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Joining School Indicator, School Effectiveness, and 
School Improvement Research: The International Perspective. 



David Reynolds, University of Newcastle, United Kingdom, 
Charles Teddlie, Louisiana State University 



Paper number one in a symposium entitled “Integrating School Indicators, School Effectiveness, 
and School Improvement Research: The Louisiana School Effectiveness and Assistance 
Pilot (SEAP)” presented at the 1998 Annual Meeting of the American Educational 
Research Association in San Diego, CA. 



Joining School Indicator, School Effectiveness, and 
School Improvement Research: The International Perspective 

Abstract: This paper will first discuss the artificial cleavage that still persists between the three 
related fields of school indicators, school effectiveness, and school improvement. This discussion 
will focus on the paradigmatic differences (e g., positivism, constructivism) that have separated 
the three fields and how these differences are now reconcilable within the paradigm of 
pragmatism, together with the utilization of mixed methods and mixed model designs. 

The sources for this paper include: Reynolds, Hopkins, and Stoll’s (1993) review linking 
school effectiveness and improvement; Stoll’s (1996) chapter on issues concerning the linking of 
school effectiveness and school improvement; Teddlie and Reynold’s forthcoming International 
Handbook of School Effectiveness Research, which includes chapters that review and integrate 
the three related fields; a chapter from that Handbook by Fitz-Gibbon and Kochan that includes a 
discussion of the relationship between school indicators and the other two fields; and Tashakkori 
and Teddlie’ s (1998) Mixed Methods and Mixed Model Studies in the Social and Behavioral 
Sciences , which reviews the denouement of the paradigm wars and the emergence of pragmatism 
and mixed model designs. 

The paper will conclude with a brief overview of the School Effectiveness and Assistance 
Pilot (SEAP) project from Louisiana, which is an ongoing three year research project associated 
with a legislatively mandated School and District Accountability program.. This project joins all 
three fields in three ongoing phases: SEAP -I, school indicators; SEAP-II, school effectiveness; 
and SEAP-HI, school improvement. This part of the paper also discusses how the School and 
District Accountability legislation that created SEAP calls for a combination of school 
accountability and improvement, which make the linking of the three fields possible. 

The authors conclude that future research programs at the state, national, and international 
levels should integrate the methodologies of the three related fields, thus allowing the 
identification of more effective/typical/less effective schools based on multiple indicators, the 
intensive assessment of schools of interest, and the development of change process models for the 
identified schools. 



Joining School Indicator, School Effectiveness, and 
School Improvement Research: The International Perspective 

I. Introduction 

There have been numerous calls for the integration of school indicator, school 
effectiveness, and school improvement efforts (e.g.. Brown, Riddell, & Duffield, 1996; Fitz- 
Gibbon, 1996a; Fitz-Gibbon and Kochan, in press; Kochan, Teddlie & Franklin, 1997; Reynolds, 
Hopkins & Stall; 1993; Stoll, 1996) over the past five years. Despite this, the SEAP project in 
Louisiana appears to be the first program of its scope (i.e., state sponsored and operating at the 
entire state level) to be implemented. 

The major factor keeping these three fields of study apart has been their separate 
evolutions as independent areas of academic specialization with distinct methods and paradigms 
that underlie them. In fact, the artificial cleavage among these fields is part of a larger distinction 
among the paradigms. 

A paradigms may be defined as the worldview or belief system that guides researchers 
(Guba and Lincoln, 1994). The importance currently attributed to paradigms in the social and 
behavioral sciences derives to a large degree from Kuhn's (1970) influential book entitled The 
Structure of Scientific Revolutions . In this book, he argues that paradigms are the models that 
are imitated within any given field, and that competing paradigms may exist simultaneously, 
especially within immature sciences (Kneller, 1984; Kuhn, 1970). 

Over the past three decades, several debates or "wars" (e.g.. Gage, 1989; Datta, 1994; 
Guba and Lincoln, 1994; House, 1994; Rossi, 1994; Tashakkori and Teddlie, 1998) have raged in 
the social and behavioral sciences regarding the superiority of one or the other of the two major 
social science paradigms or models. These two models are known alternately as the 
positivist/empiricist approach or the constructivist (interpretivist)/ phenomenological orientation 
(e.g., Cherryholmes, 1992; Guba and Lincoln, 1994). 

The positivist paradigm underlies what are called quantitative methods, while the 
constructivist paradigm underlies what are called qualitative methods (e.g., Guba and Lincoln, 
1994; Howe, 1988; Lincoln and Guba, 1985). Therefore, the debate between these two 
paradigms has sometimes been called the qualitative-quantitative debate (e.g., Reichardt and 
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Rallis, 1994). In fact, the abbreviations QUANS (for those preferring the quantitative point of 
view) and QUALS (for those preferring the quantitative point of view) have been utilized in 
describing participants in these debates or "wars" (e.g., Creswell, 1995; Morse, 1991). 

The three fields under study may be arrayed on a continuum from the most qualitatively 
oriented to the most quantitatively oriented: school indicators (almost entirely quantitative) to 
school effectiveness (began as quantitative, now more mixed) to school improvement (primarily 
qualitative since the 1980s). There has been some rapprochement between school effectiveness 
and school improvement, but school indicators has remained a separate field. The next section of 
this paper explores the linking, or lack thereof, of these three fields. 

n. Paradigmatic Differences Among the Three Fields: School Indicators, School 
Effectiveness, and School Indicators 

A. The Paradigm of School Improvement 

According to Reynolds, Hopkins, and Stoll (1993), school improvement in the United 
States, the United Kingdom and internationally in the 1960's and 1970's displayed a number of 
paradigmatic characteristics associated with the positivist/empiricist tradition. It was linked as an 
enterprise to a technological view of school improvement, in which innovations were brought to 
schools from outside of them and then introduced ‘top down’. The innovations were based upon 
knowledge produced by persons outside the school, the focus was on the school's formal 
organization and curriculum, the outcomes were taken as given, and the innovation was targeted 
at the school more than the individual practitioner. The whole improvement edifice was based 
upon a positivistic, quantitative evaluation of effects. The world-wide failures of this model of 
school improvement to generate more than very partial take up by schools of the curricula or 
organizational innovations became an established finding within the educational discourse of the 
1970’s, explained widely as due to a lack of teacher “ownership”. 

Out of the recognition of this failure came the new improvement paradigm of the 1980’s, 
which is still reflected in much of the writing on school improvement that is current and in 
evidence today. This new movement celebrated a “bottom up” approach to school improvement, 
in which the improvement attempts are owned by those at the school level, although outside 
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school consultants or experts can put their knowledge forward for possible utilization. This new 
approach tended to celebrate the “folk-lore” or practical knowledge of practitioners rather than 
the knowledge base of researchers, and focused on changes to educational processes rather than 
to school management, or organizational features which were regarded as reified constructs. It 
wanted the outcomes or goals of school improvement programs to be debated and discussed, 
rather than accepted as given. Those working within this constructivist/phenomenological 
paradigm also tended to operate at the level of the practitioner rather than at the level of the 
school, with a qualitative and naturalistically orientated evaluation of the enterprise being 
preferred to quantitative measurement. The improvement attempt was “whole school” oriented 
and school based, rather than outside school or course based (see Reynolds, 1988). 

Thus, Table 1 summarizes the evolution of the school improvement tradition from a 
primarily positivist/empirist paradigm in the 1960's/l 970's to a primarily constructivist/ 
phenomenological in the 1989's/1990's. It could be argued that this evolution has continued intop 
the later 1 990's, and that school improvement is now envisioned by some as a simultaneous “top 
down” and “bottom up” process (e.g., Murphy, 1992). This is certainly the case with the ongoing 
SEAP school improvement efforts, which are tied to an accountability legislation that puts a 
premium on quantitative achievement data. 



Table 1 

Characteristics of Two School Improvement Paradigms 



Characteristic 


1960’s 


1980’s 


Orientation 


“Top Down” 


“Bottom Up” 


Knowledge Base 


Elite Knowledge 


Practitioner Knowledge 


Target 


Organization or 
Curriculum Based 


Process Based 


Outcomes 


Pupil Outcome Oriented 


School Process Oriented 


Goals 


Outcomes as Given 


Outcomes Problematic 


Focus 


School 


Teacher 


Methodology of 
Evaluation 


Quantitative 


Qualitative 


Site 


Outside School 


Within School 


Focus 


Part of School 


Whole School 



Note . This table was taken from Reynolds, Hopkins, and Stoll’s (1993), p. 40. 
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B. The Paradigm of School Effectiveness 

As indicated by Reynolds, Hopkins, and Stoll (1993), the school effectiveness research 
paradigm has a very different intellectual history and has exhibited a very different set of core 
beliefs concerning operationalization, conceptualization and measurement by comparison with the 
changing approaches of the school improvers. It has been strongly committed to the use of 
quantitative methods, since many researchers were concerned to refute the “schools make no 
difference” hypothesis advanced by Coleman et al (1966) and Jencks et al (1972) by utilizing the 
same conventional methods of empirical research as their perceived opponents had utilized. Many 
researchers have also believed that teachers, especially North American ones, would pay more 
attention to work conducted within the quantitative paradigm. 

School effectiveness researchers have also been primarily concerned with pupil academic 
and social outcomes, which is not surprising given the political history of school effectiveness 
research in the United States, where it has grown and built on the beliefs of Ron Edmonds and his 
associates that ‘all children can learn’. Processes within schools only have an importance within 
the school effectiveness paradigm to the extent that they affect outcomes - indeed, one ‘back 
maps’ with the paradigm from outcomes to process. The school effectiveness paradigm 
furthermore regards pupil and school outcomes as fundamentally unproblematic and as given. 
Indeed, only a limited range of outcomes are used, reflecting the acceptance of ‘official’ 
educational definitions of the school as an academic institution. School effectiveness researchers 
indeed often talk of a ‘good’ or ‘excellent’ school as if that were unproblematic. 

The school effectiveness paradigm is organizationally rather than process based in terms of 
its analytic and descriptive orientation, preferring to restrict itself to the more easily quantifiable 
or measurable. As an example, Fullan's (1985) process factors such as ‘a feel for the process of 
leadership’ or ‘a guiding value system’, or ‘intense interaction and communication’ are largely 
eschewed in favor of organizationally and behaviorally oriented process variables such as ‘clear 
goals and high expectations’ and/or ‘parental involvement and support’. Additionally, the focus 
within the school improvement paradigm on the attitudinal, and on personal and group “‘inner 
states”, is replaced within school effectiveness research by a focus on the more easily measured 
behavior of persons. 
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A last couple of differences are also clear. School effectiveness research has customarily 
celebrated the importance of a very limited range of outcomes, mostly academic and mostly 
concerned with the acquisition of basic skills. Indeed, virtually all the early American work 
focused upon academic achievement virtually exclusively. 

School improvement research by contrast has often conceptualized outcomes more 
broadly. Often indeed in the British tradition the improvement attempt or project was to debate 
the ‘possible’ goals of education, as against the limited ‘official’ goals, as part of the process of 
securing improvement. In most accounts of their processes (e.g. Hopkins, 1987), it seems that 
multiple goals form the intellectual base of school improvement. 

Lastly, school effectiveness differs from school improvement in that it is concerned to 
celebrate the ‘end state’ of describing what it is that schools which are effective are actually ‘like’, 
whereas school improvement has been more concerned to discover what it is that has been done 
to bring schools to that state. The orientation of school effectiveness has been a ‘static’ one 
concerned with the ‘steady state’ of effectiveness; the orientation of school improvement has been 
a ‘dynamic’ one, focusing upon ‘change over time’. 

In the last several years, the voices calling for links between school effectiveness and 
school improvement have reached something of a chorus. Stoll (1996) argues that 'if practitioners 
can see and make links between school effectiveness and school improvement, surely it is time for 
researchers studying the two areas to do the same and to work with schools to develop a deeper 
and more meaningful understanding of the research and its implications for practice'. Hopkins 
(1996) argues that 'one of the most encouraging recent developments in the area of school 
effectiveness and school improvement is the seriousness with which the confluence of these two 
streams of enquiry is being taken'. 

C. Comparisons Between School Effectiveness and School Improvement 

From the outline of the two paradigms above, it can be seen that the disciplines of school 
effectiveness and school improvement have come from very different places intellectually, 
methodologically, and theoretically. Differences in these approaches are depicted in Table 2. 



Table 2 

The Separate Traditions of School Effectiveness and School Improvement 



Traditional School Effectiveness Model 


School Improvement in the 1980s 


Focus on Schools 


Focus on Individual Teachers or Groups of 
Teachers 


Focus on School Organization 


Focus on School Processes 


Data Driven, with Emphasis on Outcomes 


Rare Empirical Evaluation of Effects of 
Changes 


Quantitative in Orientation 


Qualitative in Orientation 


Lack of Knowledge about How to Implement 
Strategies 


Concerned with Change in Schools 
Exclusively 


More Concerned with Change in Pupil 
Outcomes 


More Concerned with Journey of School 
Improvement than Its Destination 


More Concerned with Schools at a Point in 
Time 


More Concerned with Schools as Changing 


Based on Research Knowledge 


Focus on Practitioner Knowledge 



Note . This table was taken from Reynolds, Hopkins, and Stoll’s (1993), p. 44. 
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While these differences have characterized the relationship between the two fields 
historically, changes have occurred over the past five to ten years. The distinction between school 
effectiveness and school improvement research have begun to blur. Examples of this “blurring” or 
merger include the following: 

(1) There are several examples from the research literature of linkages between programs 
between the fields of school effectiveness and school improvement (see section below for 
examples), 

(2) There has been tacit acceptance of the linkage found in the naming of important 
entities such: as the International Congress for School Effectiveness and School Improvement 
(ICSEI), the AERA SIG for School Effectiveness and School Improvement, and the School 
Effectiveness and School Improvement journal. 

(3) a 1996 book entitled Merging Traditions: The Future of Research on School 
Effectiveness and School Improvement (Gray, Reynolds, Fitz-Gibbon, and Jesson, 1996) provided 
the intellectual underpinning for a close collaboration between the two fields, plus concrete 
examples of such interaction between scholars in the two fields. 

Stoll (1996) summarized the impetus for more collaboration between the two fields: 

If practitioners can see and make the links between school effectiveness and school 

improvement, surely it is time for researchers studying the two areas to develop a deeper 

and more meaningful understanding of the research and its implications for practice. (Pp. 

51-52) 

According to some writers, school effectiveness research occupies an increasingly 
“centrist” position as it synthesizes and incorporates different intellectual traditions from different 
countries. In the forthcoming Handbook of School Effectiveness Research, the editors (Teddlie 
and Reynolds, in press) contend that there are, indeed, three traditions within school effectiveness 
research: school effects research (primarily quantitative), effective schools research, and school 
improvement research: 

(1) School effects research (primarily quantitative) comprised, for the most part, of input- 
output studies focusing on the scientific properties of school effects. The field has grown 
increasingly complex over time, evolving from relatively simple, input-output models at the level 
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of the school or district to elaborate, multi-level designs utilizing sophisticated statistical 
techniques such as hierarchical linear modeling (HLM) and canonical correlation. In sum, school 
effects studies tend to involve the manipulation of large datasets by researchers who have never 
visited the campuses whose performance they weigh. 

(2) Effective schools research (mixed model studies of differentially effective schools) is 
the logical extension of school effects research in that it represents the expansion of simple 
input/output school effects models to include measures of school context and process. In 
contrast to school effects research, the classic effective schools study utilizes both quantitative 
and qualitative methods to collect and analyze data gathered in the course of intensive, site-based 
research. 

(3) School improvement research (as noted above, primarily qualitative in nature) is the 
“action research” arm of school effectiveness research. Characteristically, school improvement 
studies apply school effectiveness principals to the development of strategies for improving 
instructional delivery at specific school sites (Reynolds, Teddlie, Creemers, Scheerens & 
Townsend, in press). 

D. The Paradigm of School Indicators 

According to Fitz-Gibbon and Kochan (in press), an education indicator is a statistic 
collected at regular intervals to track the performance of an education system. Shavelson, 
McDonnel & Oakes describe an education indicator as an “individual or composite statistic that 
relates to a basic construct in education and is useful in a policy contexff 1989, p. 5). To 
discourage researchers from labeling every bit of archival data collected “an indicator,” the U.S. 
Office of Educational Research and Improvement (OERI) has gone so far as to caution that: 

statistics qualify as indicators only if they serve as gauges', that is, if they tell a great deal 
about the entire system by reporting the condition of a few particularly significant features. 
For example, the number of students enrolled in schools is an important fact about the size 
of the educational system, but it tells little about how well the system is functioning. On 
the other hand, a statistic that reports the proportion of secondary students who 
completed advanced study in mathematics provides useful information about the level that 



students are participating and achieving in that subject... and can appropriately be 

considered an indicator” (OERI, 1988, p.5). 

There are three types of indicators: outcome, intake, and process (e.g.. Blank, 1993; Fitz— 
-Gibbon and Kochan, in press; Willms, 1992). While most educational indicators in these three 
areas are quantitative in nature, qualitative process indicators could be developed in the areas of: 

(1) outcome indicators - quality of life, and 

(2) process indicators - alterable classroom variables and alterable school management 
variables. 

The history of the school indicators literature in the United States, as well as the school 
effectiveness literature, starts with the Coleman Report (Kochan, 1998). The Coleman Report’s 
greatest legacy may be that it whetted policy makers’ appetites for more detailed and precise 
information on education, and spurred the development in the U.S. of these two research 
disciplines (school effectiveness, school indicators) dedicated to meeting the perceived research 
need. 

Interestingly, British school effectiveness research had a similar etiology (Reynolds, et al., 
1994; Teddlie, 1994). The Plowden Committee Report (1967) in the U.K. reached conclusions 
similar to those of the Coleman Report regarding the relationships among student background, 
schooling, and achievement. Early school effectiveness studies conducted in the U.K (e.g., 
Reynolds, 1976; Rutter, Maughan, Mortimore, and Ouston , 1979) were aimed at demonstrating 
the “the outcomes of individual schools were not determined by ... their intakes of pupils” 
(Reynolds, 1993, p.3). Similarly, school indicator work in the U.K. was partially a reaction to 
these early input-output studies of school effects. As Willms noted (1992, p.28): “Monitoring 
systems are based on a theory about how schooling ‘inputs’ cause schooling ‘outputs.’” 

Over the course of the past 30 years, the fields of school effectiveness and school indicator 
research have traveled down parallel paths. During that time, school indicator researchers have 
adhered more closely to scientific research models than have their school effectiveness colleagues, 
and have tended to rely on increasingly sophisticated multi-level statistical procedures to improve 
the precision and detail of their findings. School effectiveness researchers have achieved similar 
success in expanding and refining our understanding of the schooling process, but have tended to 
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do so through a greater emphasis on mixed method designs that combine site-based qualitative 
and quantitative research. From time to time, researchers in both fields have been distracted from 
their original task (describing and explaining the education process) by forays into action research. 
For school effectiveness researchers, the distraction has been school improvement; for school 
indicator research, it has been school accountability (Kochan, 1998). 

E. The Relationship Between the School Indicators and School EfTectiveness/School 
Improvement Research Fields 

While there has been a partial merger between the school effectiveness and school 
improvement literatures, this has not happened to any great extent with school indicators. At this 
point in time, the school indicators literature is an independent field that has few linkages with 
either the school effectiveness or school improvement areas. There are a number of reasons for 
this including: 

(1) The continued emphasis in the school indicators literature on accountability 
considerations, often mandated by the state. State mandates often preclude the type of research 
that would involve school effectiveness or school improvement constructs and methods. 

(2) The continued reliance in school indicators research on quantitative methods almost 
exclusively. 

(3) The reluctance of researchers in the school indicators area to get involved directly 
with school improvement efforts. 

The remainder of this section expands upon these general points and discusses some 
potential areas of rapprochement among the three research areas. 

1. The Fields of School Indicators and School Effectiveness 

Despite their common origins and similar aims, the two fields of school indicators and 
school effectiveness remain largely independent today. Much of the difference between school 
effectiveness and school indicator research concerns the methodological and logistical distinctions 
between the two fields. 



A primary characteristic of school indicators systems is the requirement for “universal” 
measures due to the accountability functions that such systems typically serve. This necessarily 
leads to an emphasis on quantitative methods in these indicator systems, since it is easier to 
construct a numeric index with standard interpretations than a qualitative one. School 
effectiveness research, on the other hand, has moved toward a mixed methodological approach 
over the past twenty years. 

Despite the overall quantitative orientation, there have been numerous calls in the school 
indicators literature for the development of school effectiveness indices and school process 
indicators that reflect school climate and/or behavioral outcomes over the past ten years (e.g.. 
Good & Brophy, 1986; Kochan, 1998; National Study Panel on Education Indicators, 1991; 
Oakes, 1989; Porter, 1991). Research also shows that policy makers and parents want access to 
such information when evaluating education quality (Jaeger, Johnson & Gomey, 1993). 
Nevertheless, there are many problems inherent in the implementation of such mixed model 
indicator systems, not the least of which is the cost and effort required to collect process data that 
accurately reflect the condition of education (Kochan, 1998). 

During the past 20 years, school effectiveness researchers have offered valuable insights 
into the contextual characteristics of schools that have high rates of student attendance and 
retention, but have primarily done so through intensive, site-based research in small samples of 
schools already identified as “effective” on the basis of achievement alone (e.g., Coleman, Hoflfer 
& Kilgore, 1982; Teddlie and Stringfield, 1993). Normative findings in this regard have been 
limited to the relatively small number of studies that have looked at schools of all types (effective, 
ineffective, and typical). 

School indicator researchers, on the other hand, have routinely gathered attendance and 
dropout statistics on very large samples of schools, but have made little progress in terms of 
translating such statistics into meaningful school process indicators. This is largely because few 
mechanisms exist for the large-scale collection of process data that are sufficiently detailed, yet 
practicable, in terms of reporting burden and cost. 

Both research limitations - the shortage of normative data derived from school 
effectiveness research and the limited depth characterized by most school indicator research - 
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could be overcome if a school performance model focusing on multiple outcomes could be 
developed that (a) utilizes principles of school indicator research, and (b) has been validated 
through site-based qualitative research in the school effects tradition (Kochan, 1998). 

2. The Fields of School Indicators and School Improvement 

The distinction between the school indicators and school improvement area may be even 
greater than that between school indicators and school effectiveness. Individuals writing in the 
school indicators area prefer to discuss the policy implications of their work, but not the direct 
implications that their work has for school improvement. Those writing in the school indicators 
area are often concerned about the potential for the misuse of their data under the guise of reform. 
For example, Willms stated: 

Statistics describing the health of educational systems can be used to demonstrate the need 
for reform arising from poor management of a previous administration, or to demonstrate 
improvements stemming from reforms of the administration in power. Some critics 
contend that analysts choose to report, depending on their political purposes, statistics 
describing absolute levels of performance, changes in levels of performance, levels or 
changes for a particular subsample of the population, or comparisons with other districts, 
states, or countries. Porter (1988) argues that performance indicators are merely a 
political tool designed to strengthen the hand of those favoring centralized control of the 
process and products of teaching. (1992, pp. 4-5). 

Fitz-Gibbon (1996a) has written about these issues, referring to the “corruptibility” of 
educational indicators, in which raw data are changed in order to alter the values reported in 
indicator systems. She contends that the greater the extent to which the indicators are used to 
publicly pressure institutions, then the greater is the need for “incorruptible indicators” (Fitz- 
Gibbon, 1996a; Fitz-Gibbon and Kochan, in press). 

Thus, there is a desire among school indicators researchers, and rightfully so, to keep their 
systems separate from whatever educational reform is ongoing at any point in time. Furthermore, 
many school indicators researchers do not believe that it is appropriate for them to try to influence 



the manner in which any school would undergo reform. These researchers believe that the 
purpose of an indicator system is to inform the faculty at a school of their current standing, not to 
guide school change. These researchers believe that each school must determine its own route to 
improvement, exclusively through the “bottom up” process described in an earlier section of this 
paper. 

Thus, there is an inclination among school indicators researchers to separate their systems 
from school improvement for two reasons: 

(1) School reform efforts tend to corrupt school indicators, and indicators should be 
made as “incorruptible” as possible. 

(2) School indicators researchers should not be involved in individual school 
improvement, since proper school change must come from the faculty at the school; that is, school 
reform has to be “bottom up”. 

3. An Emerging Dialogue Between School Indicators and the Other Fields 

There are some indications that a dialogue may be emerging between the fields of school 
indicators and school effectiveness/school improvement. 

(1) Chapters on school indicators are beginning to appear in books devoted primarily to 
school effectiveness. For example, there was a Fitz-Gibbon (1996b) chapter on monitoring 
school effectiveness in Merging Traditions: The Future of Research on School Effectiveness and 
School Improvement (Gray, Reynolds, Fitz-Gibbon, and Jesson, 1996). Also, there is a chapter 
on school indicators (Fitz-Gibbon and Kochan, in press) in the forthcoming International 
Handbook of School Effectiveness Research. 

(2) There are some ongoing research projects that attempt to integrate school indicators 
research with the two other traditions. For example, Kochan (1998) recently conducted 
dissertation research that involved a school indicator model focusing on multiple outcomes 
(achievement and behavioral), that also employed site-based qualitative research in the school 
effectiveness research tradition. The ongoing SEAP project described in this symposium is an 
example of a research program that integrates all three approaches. 
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(3) The educational realpolitik in several states in the U.S. does not allow for the 
distinction between school indicators and school reform described in the previous section. For 
example, the Louisiana Legislature in its 1997 session passed a bill that addressed both school 
accountability and school improvement simultaneously. This type of omnibus legislation demands 
an integrated approach in which the school accountability (indicator) model is inextricably linked 
to the determination of individual school effectiveness and to the promotion of individual school 
improvement. 

While other states (e.g., Illinois, Kentucky, Maryland) have also addressed school 
accountability and school improvement simultaneously, the Louisiana situation is somewhat 
unique. In most states, the individuals who work on the educational indicator systems (typically 
psychometricians or computer experts) are not the same people who work in improving schools 
(education specialists, with expertise in school effectiveness and school improvement). During 
the SEAP project, the Louisiana Department of Education (LDE) reorganized structurally around 
the function of school improvement. A high percentage of the LDE staff was trained to conduct 
the SEAP-II intensive school assessment process. Almost all of those who were trained went on 
site visits in SY 1996-97 or SY 1997-98. Many of those who went on site visits wrote parts of 
the SEAP-II report, which included a section on school improvement. Many of those individuals 
will be directly involved in the program’s school improvement activities (SEAP-III). 

Several of those individuals who have been involved with SEAP-II and SEAP-III are also 
engaged in the development of the educational indicator system (SEAP-I) that will determine 
which schools are not meeting state standards. Thus, there are a number of individuals in the 
LDE who will work on the school indicator system (SEAP-I), the intensive assessment process 
(SEAP-II), and the school improvement efforts (SEAP-III). These integrated staff activities lead 
to a high degree of program cohesion. 
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ITT. The Paradigm Wars and their Denouement 



The following section presents a more general discussion of the paradigm differences that 
underlie these differences among the fields of school indicators, school effectiveness and school 
improvement. It is based on Tashakkori and Teddlie’s (1998) Mixed Methods and Mixed Model 
Studies in the Social and Behavioral Sciences, which reviews the denouement of the paradigm 
wars and the emergence of pragmatism and mixed model designs. 

Throughout this section , the reader should recall that the school indicators field probably 
prescribes more to the post-positivistic (quantitative methods)) point of view at this time, while 
the school improvement field leans more toward the constructivist (qualitative methods) point of 
view. School effectiveness research currently has more of a pragmatist orientation (mixed 
methods). 

J 

A. The Paradigm Wars 

The paradigm wars have been fought across several "battlefields" concerning important 
conceptual issues, such as the "nature of reality" or the "possibility of causal linkages". No 
discipline (or subdiscipline) in the social or behavioral sciences has avoided manifestations of 
these paradigm wars. Datta (1994) called the participants in such wars "wrestlers"; we prefer to 
use the more genteel term "warriors". The "warriors" have included some of the best known 
scholars in their fields. 

An example of the "warriors" from Education are Lincoln and Guba (1985), who have 
contended that the tenets of positivism and the quantitative methodology and data sources that 
accompany that paradigm have been discredited, and that constructivism and qualitative methods 
are in ascendance. Smith and Heshusius (1986), also writing in the field of Education, suggested 
"shutting down" the dialogue between the two camps, saying that their incompatibility made 
further dialogue unproductive. This point of view has been called the incompatibility thesis. 

The paradigm debates have primary value now within the history of philosophy in the 
social sciences, since many active theorists and researchers have adopted the tenets of paradigm 
relativism, or the use of whatever philosophical and/or methodological approach works for the 



particular research problem under study (e.g., Howe, 1988; Reichardt and Rallis, 1994). Even 
some of the most noted warriors (i.e., Guba and Lincoln, 1994) have signaled an end to the wars 
stating: 

The metaphor of paradigm wars described by Gage (1989) is undoubtedly overdrawn. 
Describing the discussions and altercations of the past decade or two as wars paints the 
matter as more confrontational than necessary. A resolution of paradigm differences can 
occur only when a new paradigm emerges that is more informed and sophisticated than 
any existing one. That is most likely to occur if and when proponents of these several 
points of view come together to discuss their differences .... (Guba and Lincoln, 1994, p. 
116). 

Table 3 presents the distinctions between positivism/post-positivism and constructivism 
that have characterized the paradigm wars. On all these issues (methods, logic, epistemology, 
axiology, ontology, causal linkages), these two points of view are directly in contrast to one 
another. These distinctions will be discussed in detail in the next section of this paper when the 
philosophy of pragmatism is introduced. 

B. The Philosophy of Pragmatism: The Rationale for Linking the Three Fields 

Recently, there have been numerous attempts in the various social and behavioral sciences 
to make peace between the two major paradigms and their methods. "Pacifists" have appeared in 
each of the disciplines stating that qualitative and quantitative methods are, indeed, compatible. 

In Education and Evaluation research (e.g., Howe, 1988; Reichardt and Rallis, 1994), authors 
have presented the so-called compatibility thesis based on a different paradigm, which some 
have called pragmatism. Thus, we may refer to these pacifists in the paradigm wars as 
pragmatists. 
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Table 3 

Comparisons of Three Important Paradigms 
Used in the Social and Behavioral Sciences 



Paradigm 


Positivism 


Post-Positivism 


Constructivism 


Methods 


Quantitative 


Primarily 

Quantitative 


Qualitative 


Logic 


Deductive 


Primarily Deductive 


Inductive 


Epistemology 


Objective Point of View. 
Knower and Known are 
Dualism. 


Modified Dualism. 
Findings Probably 
Objectively 
"True". 


Subjective Point of Vie 
Knower and Known ar 
Inseparable 


Axiology 


Inquiry is Value Free. 


Inquiry Involves Values, 
But They May Be 
Controlled. 


Inquiry is Value Boun 


Ontology 


Naive Realism 


Critical or 

Transcendental Realism 


Relativism 


Causal Linkages 


Real Causes Temporally 
Precedent to or 
Simultaneous with 
Effects 


There are Some Lawful, 
Reasonably Stable 
Relation- 
ships Among Social 
Phenomena. These May 
be Known Imperfectly. 


All Entities 
Simultaneously 
Shaping Each Other. It 
Impossible to Distingui 
Causes from Effects. 



Note . This table was taken from Tashakkori and Teddlie (1998) Mixed Methods and Mixed 
Model Studies in the Social and Behavioral Sciences, p. 23 . 
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Pragmatically oriented theorists and researchers now often speak of mixed methods (or 
mixed methodology or methodological mixes), which contain elements of both the quantitative 
and qualitative approaches (e.g., Brewer and Hunter, 1989; Patton, 1990.) For instance, Greene, 
Caracelli, and Graham (1989) presented 57 studies that employed mixed methods and described 
design characteristics of these mixed studies. 

This detente in the paradigm wars has been positive for further research development in 
many fields of study, since most researchers now concentrate on doing research using whatever 
method is appropriate, instead of debating about methods or using one method exclusively. 
Nevertheless, the pragmatists have often employed imprecise language in their methodological 
discussions, utilizing some rather generic terms (e g., mixed methods) to connote several different 
ways of conducting a study or a series of studies. Datta recently referred to what she called 
"mixed-up models", that derived from the "... lack of a worldview, paradigm, or theory for 
mixed-model studies", concluding that "such a theory has yet to be fully articulated" (1994, p. 

59). 

C. Comparisons Among the Paradigms 

Table 4 contains a presentation of what we consider to be the primary distinctions among 
four major paradigms used in the social and behavioral sciences: positivism, postpositivism, 
pragmatism, and constructivism. It is interesting to note that theorists delineating between 
positivism (including postpositivism) and constructivism (or interpretivism) typically do not 
include pragmatism as a third point of comparison, even though those two competing points of 
view do not exhaust the paradigmatic possibilities. As Howe (1988) notes: 

This seems to be a serious omission, for pragmatists were largely responsible for bringing 
down positivism and would clearly reject the forced choice between the interpretivist and 
positivist paradigms (p. 13). 
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Table 4 

Comparisons of Four Important Paradigms Used in the 
Social and Behavioral Sciences: The Addition of Pragmatism 



Paradigm 


Positivism 


Post-Positivism 


Pragmatism 


Constructivism 


Methods 


Quantitative 


Primarily 

Quantitative 


Quantitative 
+ Qualitative 


Qualitative 


Logic 


Deductive 


Primarily 

Deductive 


Deductive + 
Inductive 


Inductive 


Epistemology 


Objective Point of 
View. Knower and 
Known are 
Dualism. 


Modified 

Dualism. 

Findings Probably 

Objectively 

"True". 


Both Objective 
and Subjective 
Points of View. 


Subjective Point 
of View. 
Knower and 
Known are 
Inseparable 


Axiology 


Inquiry is Value 
Free. 


Inquiry Involves 
Values, But They 
May Be 
Controlled. 


Values Play a 
Large Role in 
Interpreting 
Results. 


Inquiry is Value 
Bound. 


Ontology 


Naive Realism 


Critical or 

Transcendental 

Realism 


Accept External 
Reality. Choose 
Explanations 
that Best 
Produce Desired 
Outcomes. 


Relativism 


Causal 

Linkages 


Real Causes 
Temporally 
Precedent to or 
Simultaneous with 
Effects 


There are Some 
Lawful, 

Reasonably Stable 
Relation- 
ships Among 
Social 

Phenomena. 

These May be 

Known 

Imperfectly. 


There May Be 
Causal Relation- 
ships, but We 
Will Never Be 
Able to Pin 
Them Down. 


All Entities 
Simultaneously 
Shaping Each 
Other. It's 
Impossible to 
Distinguish 
Causes from 
Effects. 



Note . This table was taken from T ashakkori and T eddlie (1998) Mixed Methods and Mixed 
Model Studies in the Social and Behavioral Sciences, p. 23. 
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1. Rejection of the Either/Or 

In Table 4 the pragmatist point of view is illustrated as rejecting the forced choice between 
positivism (including postpositivism) and constructivism with regard to methods, logic, and 
epistemology. In each case, pragmatism rejects the either/or of the incompatibility thesis and 
embraces both points of view. Nielsen (1991, p. 164) suggests that pragmatism is a "reactive, 
debunking philosophy" that argues against dominant systematic philosophies, making mocking 
critiques of metaphysical assertions such as "the grand Either/Or". 

When choices come down between qualitative or quantitative methodology, 
postpositivists typically prefer experimental design (or variants thereof such as quasi-experimental 
designs or causal comparative designs), due to their preoccupation with causality and internal 
validity (e.g.. Cook and Campbell, 1979). Similarly, constructivists prefer their own methods and 
dutifully distinguish the differences in methodological orientations. 

Pragmatists, on the other hand, believe that either method is useful, choosing to use the 
broad array of all available quaitative and quantitative methods. Decisions regarding the use of 
either qualitative or quantitative methods (or both) depend upon the research question as it is 
currently posed and the phase of the research cycle that is ongoing. 

2. The Research Cycle: Utilizing Both Inductive and Deductive Logic 

Research on any given question at any point in time falls somewhere within a cycle of 
inference processes, often referred to as the research cycle or the chain of reasoning (e,g., 
Krathwohl, 1993) or the cycle of scientific methodology. The cycle may be seen as moving from 
grounded results through inductive logic to general inferences, then from those general inferences 
(or theory) through deductive logic to predictions to the particular (a priori hypotheses). . 

At some points during the research process, it is likely that both types of inferences and 
methods will be used simultaneously. Pragmatists accept that they will use both inductive and 
deductive logic in the course of conducting research on a question in which they are vitally 
interested. 



3. The Use of Both the Subjective and the Objective Points of View 

Again the black and white contrast of the incompatibility thesis (either a dualism or a 

singularity, either an objective or a subjective point of view) is challenged by the pragmatists' 
contention that researchers may be both objective and subjective in epistemological orientation 
over the course of studying a research question. In this case, it is more reasonable to think of a 
continuum than two opposing poles: at some points, one may be more "subjective", while at 
others more "objective"; at some points the knower and the known must be interactive, while at 
others, one may more easily stand apart from what one is studying. If one allows the researcher 
the use of both qualitative and quantitative methodological tools, then this embrace of both the 
subjective and objective points of view is inevitable. 

4. The Roles of Values (Axiology) 

As indicated in Table 4, positivists believe that inquiry is value free, while constructivists 
believe that inquiry is value bound. Postpositivists realize that values play an important role in 
inquiry, but they also believe that it is possible to control the degree to which values influence 
results and interpretations. Postpositivists acknowledge both the value-ladenness and the theory- 
ladenness of facts (Reichardt and Rallis, 1994). Despite this recognition (and to a large degree 
because of it), postpositivists (e.g.. Cook and Campbell, 1979) have devoted considerable effort 
to developing methods whereby the internal and external validity of their conclusions can be 
enhanced. 

Pragmatists believe that values play a large role in conducting research and in drawing 
conclusions from their studies, and they see no reason to be particularly concerned about that 
influence. Pragmatists decide what they want to research, and this choice necessarily involves 
their personal value systems; that is, they study what they think is important to study. They then 
study the topic in a way that is congruent with their value system, including variables and units of 
analysis that they feel are the most appropriate for the research. They also conduct their studies in 
anticipation of results that are congruent with their value system. This description of the way in 
which researchers conduct their research has great face validity; that is, it seems to describe the 
way that researchers in the social and behavioral sciences actually conduct their studies, especially 



research that has important social consequences. 



5. Considerations of Ontology and Causality 

The most controversial issues relating to the positivist/post-positivist, constructivist, and 
pragmatist points of views concern ontology and causality (the last two rows in Table 4). Guba 
and Lincoln (1994) consider differences concerning the nature of reality (ontology) to constitute 
the defining distinction between positivism/postpositivism and constructivism. Guba and Lincoln 
(1994) and Miles and Huberman (1994) define the following types of realism: 

(1) Naive realism - there is an objective, external reality upon which inquiry can agree 
(Guba and Lincoln, 1994, p. 1 1 1). As indicated in Table 4, this position is associated with 
positivists. 

(2) Critical realism - there is an objective reality, but it can be understood only imperfectly 
and probabilistically. This position is associated with postpositivists. 

(3) Transcendental realism - the belief that social phenomena exist in the objective world, 
and that there are some "lawful reasonably stable relationships" among them (Miles and 
Huberman, 1994, p. 429). This statement is an alternative expression of the postpositivist 
position. 

(4) Ontological Relativism - there are multiple social realities that are products of human 
intellects and that may change as their constructors change (Lincoln and Guba, 1994, p. 111). 

This position is associated with constructivists. 

The major difference between the positivists/postpositivists and the constructivists on the 
nature of reality concerns the existence of an objective, external reality: the positivists/ 
postpositivists believe that such an objective reality exists, the constructivists believe that only 
multiple, subjective realities exist. 

The pragmatist point of view regarding reality consists of two parts: 

(1) There is an external world independent of our minds (Cherryholmes, 1992, p. 14). 
Thus, the pragmatists agree with the positivists/postpositivists on the existence of this external 
reality. 

(2) On the other hand, pragmatists deny that Truth can be determined once and for all. 



They also are unsure if one explanation of reality is better than another. According to 
Cherryholmes (1992), the pragmatists' choice of one explanation over another "... simply means 
that one approach is better than another at producing anticipated or desired outcomes, (p. 15) 

Notions regarding causal relationships follow from these ontological distinctions. In 

short: 

(1) Postpositivists believe that there are some lawful, reasonably stable relationships 
among social phenomena that may be known imperfectly, or probabilistically. 

(2) Pragmatists believe that there may be causal relationships, but that we will never be 
able to pin them down. 

(3) Constructivists believe that all entities are simultaneously shaping each other and that 
it's impossible to distinguish causes from effects. 

6. The Intuitive Appeal of Pragmatism 

Given all these considerations, pragmatism appears to be the best paradigm for justifying 
the use of mixed methods and mixed model studies (Howe, 1988). Pragmatism is appealing: 

(1) because it gives us a paradigm that philosophically embraces the use of mixed method 
and mixed model designs; 

(2) because it eschews the use of metaphysical concepts (Truth, Reality) that have caused 
much endless (and often useless) discussion and debate; and 

(3) because it presents a very practical and applied research philosophy: study what 
interests and is of value to you, study it in the different ways that you desire, and utilize the results 
in ways that can bring about positive consequences within your value system. 

D. The Dictatorship of the Question 

Pragmatists consider the research question to be more important than either the method 
they utilize, or the worldview that supposedly underlies the method. Most good researchers 
prefer addressing their research questions with any methodological tool available, utilizing the 
pragmatist credo of "what works" (e.g., Cherryholmes, 1992; Howe, 1988; Rorty, 1982). For 
most researchers committed to the thorough study of a research problem, method is secondary to 



the research question itself, and the underlying worldview hardly enters the picture, except in the 
most abstract sense. 

Our best scholars have always been more interested in investigating the questions that they 
have posed than in the specific methodologies that they employ, and the paradigms that underlie 
these methods. For instance, shortly after the recent death of pioneering methodologist Donald T. 
Campbell, Gene V. Glass eulogized him in a message sent out over the World Wide Web: 

The method must follow the question. Campbell, many decades ago, promoted the 
concept of triangulation - that every method has its limitations, and multiple methods are 
usually needed. 

IV. Examples from the Research Literature of Linkages Among the Fields 

A. The Blending of the School Effectiveness and School Improvement Traditions 
The previous section presented the philosophical underpinnings for linking the fields of 
school effectiveness, school improvement, and school indicators in integrated programs of study, 
under the banner of pragmatism and using mixed methods. The following section will present 
examples of integrated research involving the school effectiveness and improvement paradigms. 

Examples in this section come from Stoll (1996), Reynolds, et al., (1993), and Reynolds, 
et al., in press. Several research programs combining the school effectiveness and school 
improvement traditions were discussed in these three chapters. 

(1) The Barclay-Calvert Project (Stringfield, et al., 1995) 

(2) The Cardiff Change Agent Study (Reynolds, Davie, and Phillips, 1989) 

(3) The Dutch National School Improvement Project (Houtveen and Osinga, 1995). 

(4) Halton’s Effective Schools Project (Stoll and Fink, 1992) 

(5) Improving the Quality of Education for All (IQEA) (Ainscow and Hopkins, 1992; 
Hopkins, et al., 1994) 

(6) Lewisham School Improvement Project 

(7) Quality Development Initiative (QDI) (Ribbins and Burridge, 1994) 

(8) Schools Make a Difference (SMAD) (Myers, 1996) 



According to Reynolds, et al. (in press), a number of intervention projects emerged in the 
late 1980’s and early 1990’s in several countries (e.g., Canada, the Netherlands, the U.S., the 
U.K.) which were neither effectiveness based nor school improvement oriented, as defined by the 
traditional limits of those disciplines. Much of this convergence between these two paradigms has 
resulted from practitioners and district policy makers borrowing from both traditions, because 
they do not share the ideological commitment to the one or other ways of working of researchers 
in the fields. Some of this blending of research approaches has also arisen through the effects of 
the ICSEI in breaking down disciplinary as well geographical boundaries. 

This failure to subscribe to an either (school effectiveness) or (school improvement) 
paradigm is the same process described in the previous section when pragmatists refused to select 
between the philosophies of positivism (quantitative methods) or constructivism (qualitative 
methods. Similar to the pragmatists who seeks “what works” and are interested primarily in the 
research question, the investigators leading these “blended” school effectiveness/school 
improvement studies seek any information or method that would lead to success in their action 
research projects. 

Sometimes the adoption of ideas from research has been somewhat uncritical; for example, 
the numerous attempts to apply findings from one specific context to another entirely different 
context when research has increasingly demonstrated significant contextual differences (Hallinger 
and Murphy, 1986; Teddlie and Stringfield, 1993). Sometimes it is clear that projects are partial 
in their adoption of material from the paradigms: some projects reflect on an understanding of 
what makes schools effective but do not have a detailed action plan to accomplish their goals, 
while other projects have celebrated the ‘core’ school improvement ideas of ownership, 
collegiality and laterality without much acknowledgment of the key areas of school process and 
organization on which to focus their attention (Reynolds, et al., in press).. 

Nevertheless, there are a number of projects in action that represent a proper blending of 
the two approaches. These blended programs have several characteristics in common: 

(1) Pupil outcomes in academic (and often social) areas are regarded as the key success 
criteria, rather than measures of teacher perception of the innovations, as was the case historically. 

(2) These outcomes are increasingly assessed by use of quantitative data, that is regarded 



as necessary to build commitment and confidence among those taking part and to measure the 
success or failure of the project initiative. 

(3) Bodies of knowledge from school effectiveness, school improvement and school 
development are used to resource programs, with a problem centered orientation being used in 
which philosophical judgements about the nature of appropriate strategies in suspended in favor 
of a “what works” (the pragmatist) approach that is distinctly non-denominational. 

(4) The learning level, the instructional behavior of teachers and the classroom level, are 
increasingly begin targeted for explicit program attention as well as the school level. This is a 
marked contrast again, with work from the 1 980's where the school was often the sole focus. 

This change mirrors a similar conversion in traditional school effectiveness research, beginning in 
the mid 1980's, when researcher began consciously including teacher effectiveness variables in 
their work (Teddlie, 1994). 

B. An Extended Example: The Barclay-Calvert Project in the United States 

The following discussion is taken from Reynolds, et al. (in press). 

Our discussion here about this project, and especially about the relationship between 
school effectiveness and school improvement, is based on a four-year evaluation (Stringfield et al, 
1995). In this project the curricular and instructional package of Calvert School, a private school 
in Baltimore (USA), is being implemented in an inner city public school (Barclay) in the same city. 
The Calvert school offers a Kindergarten to Grade Six day school program to a predominantly 
highly affluent clientele in Baltimore. Since early this century, the school has also offered a highly 
structured, certified home study curriculum. Procedures for the teachers’ provision of lessons, 
parent/school contact, and evaluation standards for each unit of the curriculum are all unusually 
specific. Each grade’s curriculum and instructional program is provided in a level of detail that 
often approximates to scripting. Moreover, the entire program places an unusually high emphasis 
on student-generated products. 

Students generate ‘folders’ of work that are regularly reviewed by their teacher, their 
parents, and the school’s administrators. According to the evaluation team, the curriculum itself 



is not revolutionary but reflects decades of high and academically traditional demands, blended 
with an emphasis on the importance of classwork and homework and an intensive writing 
program. All the work undertaken by students reflects the characteristics of effective teaching 
linked together with a high achievement level of the intakes. It is thus easy to understand why the 
results on norm-referenced achievement tests are very good. Virtually every grade of the select 
population of Calvert students score above the 90th percentile when compared to public, elite 
suburban and private school norms. 

By contrast Barclay School is a kindergarten through eighth grade public school in 
Baltimore. The population served by Barclay is 94 percent minority. Nearly 80 percent of the 
students attending Barclay School receive free or reduced price lunch. The neighborhood is one 
of old factories and terraced houses. The school serves some families led by drug dealers, 
prostitutes, or impoverished graduate students at Johns Hopkins University; however, the great 
majority of students are the children of working class or unemployed African-Americans, often 
single-parent families. In the late 1980’s it became clear that the school’s achievement scores 
were poor, as well as the attendance rate and levels of student discipline in the classrooms and 
school halls. So, Barclay could be seen as having the typical problems of an inner-city American 
school. 

The Principal at Barclay became interested in the Calvert School program and the two 
schools developed a proposal to implement the entire Calvert program at Barclay school. The 
program was to be implemented by the Barclay staff. The staff would receive training in the 
Calvert philosophy, materials, and instructional system. Ongoing staff development would be 
provided by a facilitator who had both public school and Calvert teaching experience. 

Implementation began in the Fall of 1990 in kindergarten at first grade. Each year one 
additional grade has been added. For two weeks each summer the Barclay/Calvert facilitator 
trains the next grade’s Barclay teachers in the Calvert philosophy, curriculum and methods. The 
facilitator spends the school year working with each succeeding group of teachers in an effort to 
maximize the chance of full program implementation. 

The key factors making for the project's success are argued to be: 

(1) Having Funding . The funding of the program by a private foundation made it possible 



to appoint a full-time co-ordinator who acted as the project facilitator. It should be noted, 
however, that the entire four-year grant has supplied less money than is the annual difference in 
total funding between a disadvantaged Baltimore City and a suburban Baltimore County school. 
One of the benefits of this was that everything needed for instruction was there on time in a 
sufficient quantity (one of the major disadvantages of Barclay school in the past). 

(2) Having Non-Fiscal Support . The funding foundation was involved in the program, 
not only in fiscal decisions, but also in regular staff meetings and in efforts to anticipate and solve 
problems. The principal was highly talented and very determined to make the program a success. 
She visited classes, attended meetings and was very active in involving parents. Furthermore, the 
Calvert co-ordinator at Barclay was very supportive with respect to the implementation and 
brought high levels of knowledge, competence and enthusiasm to the program implementation. 
Another source of support was Calvert School itself. The principal remained a supporter of the 
project; he assisted Barclay School, visited classrooms and repeatedly made himself available for 
consultations. Finally, the parents of Barclay were very actively involved at every stage of the 
project. 

(3) Having an Achievable Plan . The Barclay/Calvert project was very methodical. 
Barclay did not attempt to implement the whole Calvert curriculum and instructional program all 
at once, but gradually, grade level by grade level. In this way it was possible to prepare teachers 
for the next grade level utilizing a cascade model. 

(4) Having a High Quality Curriculum . The curriculum itself involves five processes that 
are extremely important for its success. 

(a) Students read a lot (that means increasing opportunity to learn). 

(b) All students produce a lot of work. 

(c) . Teachers check students’ work and also correct all their own work. 

(d) Student folders are read by the co-ordinator, the principal or the Calvert 
administrator every month. Therefore, student monitoring is very strong. 

(e) The student folders are sent home every month, which increases parent 
involvement in what students do and also in checking homework. 

(5) Having Positive Teacher Attitudes . It is striking in Calvert School that teachers are 
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highly confident that Calvert students can excel academically. It was one of the major changes of 
the implementation of the Calvert program in Barclay that the teacher attitudes changed also at 
Barclay. They were convinced that the Calvert program was helping them to teach more, teach 
better, and to help more children perform at higher levels. These are not of course unrealistic 
expectations, since the teachers have good grounds for their optimistic beliefs, because they have 
received meaningful, ongoing support and because they were observing enhanced results. 

(6) Emphasizing Classroom Instruction . Little changes in students’ academic 
achievement unless something changes between the student, the curriculum and instruction. The 
Calvert program is reaching towards the student, for which there are several reasons. 

First, the curriculum has been put in place. The curriculum demands high fidelity 
implementation. Second, a high quality instructional system is also in place. Almost all the 
teachers have made significant changes in their teaching and have become more effective. In 
classroom observation it turned out that the student ‘on-task’ rates in the Calvert/Barclay classes 
were often very high and that teachers reported that given high quality instruction and 
instructional support, the Barclay students were responding well to the raised demands. In 
summary, the Calvert program has enjoyed a far better chance of achieving full implementation 
than most novel programs in school systems. 

In the evaluation of the program, student testing was also included. The data from norm- 
referenced achievement testing programs indicate that students in the program are achieving 
academically at a rate significantly above their pre-program Barclay School peers. This finding is 
consistent across Reading, Writing, and Mathematics, and is particularly striking in writing. The 
Barclay students are also making progress not only in specific areas of academic content, but also 
in the ability to integrate new material and absorb new knowledge. Additional data indicate that 
the Barclay/Calvert project has reduced student absences, reduced student transfers from the 
school, greatly reduced the number of students requiring Chapter 1 services, reduced referrals to 
and diagnosis of ‘learning disablement’, eliminated disciplinary removals and increased the number 
of students found eligible for the Gifted and Talented Education (GATE) program. Taken 
collectively, the diverse measures indicate a very successful school improvement project, which 
above all has been reliably implemented. 



V. Linkages Between School Indicators, School Effectiveness, and School Improvement 
Through SEAP 

As noted earlier, the School Effectiveness and Assistance Pilot is a three year study that 
joins all three fields in three ongoing phases: SEAP-I, school indicators; SEAP-II, school 
effectiveness; and SEAP-m, school improvement. Due to the high degree of integration among 
job functions in the newly restructured LDE, SEAP appears to be the first, large scale, state 
mandated program to integrate all three fields of study. This is a function of three factors: 

(1) the enabling legislation, to be described below, which called for school accountability 
and school improvement; 

(2) the restructuring of the LDE into a unit, whose primary goal is school improvement, 
and 

(3) the development of a management team of state bureaucrats and university personnel, 
who conceptualized the three aspects of the project as integrated and highly interdependent. 

While reading the following section, it is important to remember that SEAP is not a 
university based research project. It is a pilot study for a legislatively mandated program (School 
and District Accountability Program). Therefore, decisions that were made regarding the design 
and implementation of SEAP had to always take into consideration the politics associated with 
state level public education. In this particular context, the major players included the legislatively 
mandated School and District Accountability Commission, the state Board of Elementary and 
Secondary Education (BESE), the Governor and his Educational Advisor, and the state 
legislature. 

A. Act 478 of the Regular Session of the 1997 Louisiana Legislature 

The education reform fervor that has consumed the United States over the past 30 years 
stimulated the creation of state education accountability systems throughout all 50 states. Though 
the precise structure of these systems varies from one locale to the next, reflecting the diversity of 
local populations and their respective education goals, the evolution of state accountability 
systems has tended to follow a general pattern (Cornett & Gaines, 1997). Louisiana has had its 
share of these reforms, as will be described later in this section. 



The most current educational reform in Louisiana was initiated through Act 478 of the 
Regular Session of the 1997 Louisiana Legislature. This act called for the development and 
implementation of a school and district accountability system, which has the following four 
purposes: 

(1) to provide for the development and implementation of a school and district 
accountability system that “requires and supports student achievement” in each public school, 

(2) to provide for the monitoring of the “quality of education in each public school”, and 
to insure that the quality of education is maintained at a level “essential for each student to receive 
a minimum foundation of education”, 

(3) to provide “clear standards and expectations” for schools and districts so that the 
assessment of their effectiveness will be understood, and 

(4) to provide information to schools and districts that will assist them in focusing on 
student achievement. 

The law called for two pilot study years (SY 1997-98, SY 1998-99) in order to prepare 
for full implementation of the state school and district accountability system in SY 1999-2000. 

The LDE had already initiated a limited pilot study of the school accountability system in SY 96- 
97, which is being continued in SY 1997-98. There are three phases of this School Effectiveness 
and Assistance Pilot (SEAP), which parallel the purposes of Act 478 (monitoring, intensive 
assessment, assistance) : 

(1) SEAP-I - the piloting of a statewide system for monitoring the effectiveness of public 
schools, 

(2) SEAP-II - the piloting of an intensive assessment of selected public schools, including 
an on-site visit, and 

(3) SEAP-III - the piloting of a school improvement program. 

The law also established the roles of BESE, the LDE , and a School and District 
Accountability Commission with regard to accomplishing its goals. The Accountability 
Commission reported its findings and recommendations to BESE in March 1998. As part of its 
duties, the Accountability Commission has been gathering information on school improvement 
plans currently in operation in other states and will provide BESE with recommendations for the 
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improvement component of the school accountability system. 



B. Past Efforts at Educational Reform in Louisiana 

1. Overview of Past Efforts at Educational Reform 

Like many states in the U.S., Louisiana has attempted a number of educational reforms 
over the past twenty-five years. If student performance on standardized tests is used as the 
criterion (e.g., student performance on the National Assessment of Educational Progress, or 
NAEP), then these past efforts have been largely unsuccessful. 

According to a recent literature review (Kochan, 1998), the progression of education 
accountability in Louisiana has tended to follow national trends: starting with a focus on 
education inputs , raising standards for instructional personnel , establishing high standards for 
pupil performance , and building to passage of a comprehensive education reform package - the 
Children First Act in 1988 - that simultaneously created a teacher evaluation program , an 
education performance indicator program to monitor school performance, and a school incentive 
program to recognize and reward high-performing schools. The Louisiana Association of 
Business and Industry published a document in 1986, entitled “Ten Years of Educational Reform. 
A Long Road to Nowhere”. This document was published two years before the passage of the 
CFA, which basically codified under one umbrella the variety of educational reforms that had been 
proposed, and occasionally enacted without adequate funding, within the state from the mid- 
1 970's. 

The Children First Act (CFA) was widely perceived to be a failed reform effort, due to the 
scraping of its centerpiece, the Louisiana Teacher Evaluation Program (Ellett, et al., 1996; Maxcy 
and Maxcy, 1993). The School and District Accountability System, established by Act 478 of the 
Regular Session of the Louisiana Legislature, is the first major statewide reform activity since the 
CFA. 

2. Education Accountability in Louisiana from 1975 through 1987: Tracking 
the National Trends 
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In keeping with national trends, Louisiana policy makers first began focusing on education 
inputs during the mid-1970's, taking steps to standardize such instructional inputs as textbooks. 
One of the major reform initiatives of this period was enacted in 1975 when the Louisiana 
Legislature authorized BESE to implement a “School Approval Process” (SAP) to ensure that all 
public schools met certain minimum standards relating to school safety, facilities, and instruction, 
such as maximum class size, the specific type of certification held by teachers, etc. Student 
outcomes did not figure into the school approval process in any way. Schools that fail to pass 
muster were given a grace period within which to correct their identified deficiencies or risk loss 
of state and federal funding. Though numerous schools have been cited over the years, 
particulary for deficiencies in certified personnel, none have been declared “unapproved” or have 
lost funding since the program’s inception, leading school districts to take the approval process 
less seriously over time. There also is a growing skepticism that an approval process that focuses 
exclusively on inputs is adequate to ensure high quality instruction, particularly since LDE 
researchers have found only weak correlations between key SAP-monitored inputs and valued 
student outcomes, such as student achievement (Crone & Franklin, 1993). 

In their efforts to improve the educational outcomes of Louisiana students, policy makers 
next turned their attention toward raising standards for instructional personnel in Louisiana 
public schools. A variety of measures were enacted during the late 1970's and early 1980's, 
establishing minimum criteria for teacher education programs (La. R.S. 17:7.2); raising standards 
for teacher certification; providing state-subsidized continuing education opportunities for 
teachers; and establishing minimum teacher salary schedules to encourage promising young 
people to enter the profession. 

Another national trend which took root in Louisiana during this period was the drive to 
improve student outcomes by raising standards for pupil performance and then monitoring 
schools’ progress toward achieving those more rigorous goals. The 1979 Louisiana Legislature 
enacted a competency-based education program that (a) mandated the adoption of statewide 
curriculum standards in required subjects, (b) directed local school districts to develop pupil 
progression plans setting local criteria for pupil promotion and placement, and (c) provided for 
the creation of a minimum standards testing program That program was repealed in 1986, 
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however, and replaced by the Louisiana Educational Assessment Program (LEAP). 

The LEAP’S primary purpose is to provide information that can be used for diagnostic 
purposes and to guide local promotion decisions. Norm-referenced tests (NRTs) were initially 
administered in grades 4, 6, and 9, while criterion-referenced tests (CRTs) were implemented in 
grades 3, 5, 7, 10, and 1 1. Coincident to the program’s creation, BESE adopted stringent new 
high school graduation requirements. Under those new guidelines, high school students were 
required to (a) pass a Graduation Exit Examination consisting of the five CRT components 
administered in grades 10 and 1 1, and (b) accumulate 23 Carnegie Units of academic credit in 
specified subject areas (Norton, Park, Christian & Eichler, 1996). 

3. The Children First Act of 1988: Teacher Accountability Plus School 
Indicators and Incentives 

The public fascination with educational accountability that swept the country after A 
Nation at Risk (1983) did not go unheeded in Louisiana (Kochan, 1998; Maxcy and Maxcy, 

1993). Louisiana elected an “Education Governor” in 1987, a reform-minded Harvard-educated 
Congressman, who promised to improve Louisiana’s schools and classrooms. Following up on 
the campaign promise that helped get him elected. Governor Buddy Roemer pushed an omnibus 
education bill through the Legislature during his first year entitled the “Children First Act.” This 
comprehensive legislation had a number of components including: the Louisiana Teacher 
Evaluation Program (LaTEP for veteran teachers), the Louisiana Teacher Internship Program 
(LaTIP for first year teachers), the Model Career Option Program (MCOP) for outstanding 
teachers, teacher and administrator pay raises, longevity steps for educators through 25 years, the 
School Progress Profiles, the School Incentive Program, etc. 

Three of these programs are particularly relevant to current educational reform in the 
state: the School Incentive Program, LaTEP, and the School Progress Profiles. The School 
Incentive Program (SIP I was designed to recognize and reward public schools that were making 
exceptional educational progress as compared to other schools of a similar type, based on student 
population factors, school environment factors, and prior achievement. During the SIP’s first and 
only year of operation, $1,000 cash awards were made to 100 schools statewide. Though never 
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repealed, the program was never funded beyond its first year in operation; its greatest contribution 
to date may be the method researchers developed to calculate a composite, school-level 
achievement score (thereafter nicknamed “SIPSCORE”), which could be used to track school 
progress from year to year (Crone and Franklin, 1993; Kochan, 1998). 

The Louisiana Teacher Evaluation Program fLaTEPt was a high-stakes professional 
accountability program whose purpose was to periodically assess the classroom performance of 
public school teachers. Though the legislation included provisions to remediate teachers in areas 
of deficiency and provide incentives for top teachers to remain in the classroom, those points were 
overshadowed by the program’s third function: to provide a mechanism for revoking the 
credentials of incompetent instructors. LaTEP drew the bitter opposition of teacher unions who 
stridently criticized the highly complicated evaluation process that was developed largely without 
teacher input. Critics so undermined the program that it was suspended after only a few years in 
operation. 

The Progress Profiles (School Report Card) Program . This component of the CFA is still 
in operation today, with the Progress Profiles Program having evolved into Louisiana’s education 
performance indicator system. In true accountability logic, “the program was founded on the 
premise that educational improvement is most successful when parents, school staff, and policy 
makers have access to information on a wide range of factors believed to influence student 
learning” (LDE, 1993). Toward that end, the program was designed with three objectives in 
mind: “to establish a database for educational planning, to increase accountability at all levels, and 
to inform the parents of school children and the general public on the condition of public 
education” (Children First Act, 1988). 

Three levels of Progress Profiles reports are published. Individualized School Report 
Cards in four formats (elementary, middle, high, and combination school) are developed on 
virtually every public school in the state, mass-produced, and distributed to public school parents 
statewide. These reports are the simplest in presentation and have presented only one year of data 
since a 1994 evaluation study demonstrated that parents and school staff had difficulty 
interpreting the indicators. A state-level report, the Louisiana Progress Profiles State Report , 
provides one year of summary, state-level information on all indicators and is designed specifically 



for state officials and policy makers. 

The most detailed and technical of the three Profiles reports is the District Composite 
Report (DCR). These individualized district-level reports provide six years of longitudinal 
information, by school, on all the schools in a district. Some indicators are presented in greater 
detail than in either the parent and policy-maker reports; all include descriptions of the data 
sources and methods used in calculating the indicators. DCRs are distributed primarily to central 
office staff, school principals, researchers and other individuals involved in school administration 
and improvement. 

C. School Effectiveness and Assistance Pilot, Phase I (SEAP-I) - the School 
Indicator Component 

This school indicator phase of SEAP involves the identification of more and less effective 
schools based on multiple indicators (school indicators). SEAP-I measures the effectiveness of 
nearly 1,500 Louisiana public schools using achievement and other performance indicators 
(dropouts, attendance). 

SEAP-I was initially an outgrowth of education finance accountability legislation in 
Louisiana, which mandated that the LDE develop academic performance measures that could be 
used to test the “equity” and “adequacy” of the state’s Minimum Foundation Program (MFP). In 
SY 1996-97, the LDE deputy superintendent for management and finance employed consultants 
from Louisiana State University (LSU) to improve upon an existing school performance model 
which used regression analysis in the calculation of school performance indicators (SEIs). 
Regression analysis or more complicated multilevel models are often used in school effectiveness 
research and in school indicator systems (e.g., Fitz-Gibbon, 1996a; Sanders & Horn, 1994; 
Willms, 1992). These models calculate school effectiveness indices (SEIs) after taking into 
consideration factors such as the socioeconomic status (SES) of students. 

In SY 1997-98, LSU consultants worked with the School and District Accountability 
Commission in examining various types of models for assessing schools’ effectiveness status. 

The results of these activities will be discussed in the third paper of the symposium. 



D. School Effectiveness and Assistance Pilot, Phase II (SEAP-I) - the School 
Effectiveness Component 

The school effectiveness phase of SEAP involves the intensive assessment of schools of 
interest identified from the results of SEAP-I. In SEAP -II, teams of observers intensively assesses 
the effectiveness of selected schools (12 in SY 1996-97, 45 in SY 1997-98) using a wide variety 
of process and product data gathered through archived sources, school site visits and the schools’ 
own internal needs assessments. The data collection is a mixed method one, with over 20 
databases being utilized. 

When the system becomes operational in SY 1999-2000, the results from SEAP-I will be 
used to determine which schools receive the intensive SEAP-II assessment process. During the 
pilot years SY 1996-97 and SY 1997-98, districts and schools volunteered to undergo the 
process. Representative samples of schools, with varying levels of achievement and of 
socioeconomic status of student body, were selected from the volunteers for the assessment. 

SEAP-II activities will be discussed in the fourth and fifth papers of the symposium. 



E. School Effectiveness and Assistance Pilot, Phase in (SEAP-m) - the School 
Improvement Component 

The school improvement phase of SEAP involves the development of change process 
models for the identified schools. When the system becomes operational in SY 1999-2000, these 
school improvement activities will be earmarked for those schools who were identified through 
the SEAP-I process and who weree intensively assessed through the SEAP-II process. 

During the pilot years SY 1996-97 and SY 1997-98, two groups of schools (Cohorts One 
and Two) received, or will receive, a set of recommendations for school improvement, which was 
supposed to be incorporated into their existing school improvement plans. These 
recommendations were part of the school’s SEAP-II report, which included a needs assessment 
completed by the schools’ faculties. 

As enacted for Cohort One (the only group to undergo improvement activities thus far) in 
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Presentation #2. Dueling Agendas: 

Louisiana’s Prescription for Balancing the Often Competing Demands 
of Education Improvement and Accountability 

Objectives: Louisiana is noted for its diverse population and distinctive political heritage. This 
paper will describe the systemic education accountability reforms currently underway in Louisiana 
and the political and social context within which those reforms are taking place. 

Perspectives: Louisiana ’s system of government is very centralized and is traditionally a top-down 
structure. The education reform efforts now underway are an attempt to break that mold. The LDE, 
under new management, is trying to (a) reconceptualize its role, moving from regulator to facilitator 
and (b) replace the current hodgepodge of performance monitoring mechanism — all artifacts of the 
piecemeal reforms of the 1970s and 1980s — with an integrated and systemic education 
accountability process. 

Conclusions/Point of View: The development y f a statewide, school accountability system is a 
process fraught with competing ideas, beliefs, philosophies, and political agendas that can stymy 
progress and subvert even the best intentions of reformers. Louisiana has embarked upon a 
progressive journey to restructure its education system by changing the LDE’s primary function 
from one of regulation to one of service. While trying to redirect resources to fulfill this new calling, 
the LDE is charged with developing a means by which schools are held accountable for student 
performance. SEAP is the LDE ’s mechanism for striking a balance between those demanding school 
closures and those striving for improvement. 



By: 

Marlyn Langley 

Deputy Superintendent for Management & Finance 
Louisiana Department of Education 

and 

Bobby Franklin, Ph.D. 

Director, Division of Planning, Analysis and Information Resources 
Louisiana Department of Education 
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INTRODUCTION TO LOUISIANA EDUCATION REFORMS 



In Louisiana, educational reforms began in the mid-1970s and have been on-going since with 
a little-bit-of-this and a little-bit-of-that, somewhat similar to making a gumbo. However, each of 
the Gumbo ingredients were being cooked in separate pots without ever being blended together in 
the Gumbo pot! One of the main reasons for the lack of coherence was an overall lack of leadership 
and a compartmentalized approach to reform. This paper will attempt to describe how the School 
Effectiveness and Assistance Pilot (SEAP) in Louisiana is helping to get all of the ingredients into 
the same gumbo pot so we can nourish Louisiana education. 

Leadership 

Public policy in Louisiana has been very centrally directed from the top by a number of 
elected bodies: the Governor, the Legislature, the State Board of Elementary and Secondary 
Education (SBESE) and the State Superintendent. At times, these bodies were in such disagreement 
that management issues entered the judicial arena, such as when the State Superintendent sued 
SBESE over the control of federal funding! Needless to say, almost every public policy regarding 
education was created and debated in the political environment. 

In 1984, the legislature changed the law to allow SBESE to appoint the state superintendent 
which was during the first legislative session of the third term of Governor Edwin Edwards. This 
action may have been politically motivated, but still required a legislative 2/3 majority vote to 
convert the office from being elected to being appointed by SBESE. The vote was so close, a 
legislator was brought in on a gurney from his hospital bed in order to vote on securing the change. 
However, this action did not go into effect until the incumbent, who had ran for office, completed 
his term. Louisiana education then underwent a four year lame-duck situation with an elected 
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superintendent who could not run again for the office. This time-period also corresponded to a 
serious drop in the price of oil, which had a tremendous economic impact on the state as a whole. 
As a result, when dramatic reductions in state funded programs were required, the lack of leadership 
contributed to fragmentation and incoherence in any remaining programs. 

Since the spring of 1988 when SBESE first appointed the state superintendent, the 
Department and SBESE have struggled to orient themselves to this change in governance styles. 
With the appointment of the current State Superintendent Cecil J. Picard on July 1, 1996, it appears 
that much of the struggling is beginning to pay off. Three months after taking office, the 
Superintendent presented to SBESE a four year action plan geared to improving student achievement 
that has served a the precursor to a five year strategic plan due to be completed June 30, 1998. The 
LDE had not had a strategic plan in at least the last twenty years. One house-keeping component 
of the action plan was the reorganization of the Louisiana Department of Education (LDE). The 
reorganization will be complete by August of 1 998 and will focus the agency on customer service 
and student achievement. 

Educational Reforms. 1975-1991 

The reforms that began in the late 1970s dealt with: student accountability by establishing 
a “basic skills” testing program along with the mandate for the state to develop the curriculum for 
every subject being taught; a local teacher evaluation program; a professional development program; 
and state school approval standards. The testing program was considered so basic, that in the mid- 
1980's it was suspended while a new testing program was created. In the late 1980s, the state 
resumed a student testing program with both criterion-referenced tests (CRT) and norm-referenced 
tests (NRT) through the Louisiana Educational Assessment Program (LEAP). This program 
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continued to be based on state established curriculum guides. The CRT testing program also 
included specific content components for high school grades 10 and 1 1 which required passage in 
order for students to graduate from high school. The curriculum standards and the testing program, 
while encouraged to be coordinated, were administered by separate bureaus in separate offices of 
the LDE. 

The Professional Improvement Program (PIPs) was active and funded for 5 years. At that 
point, the state was paying teachers who had taken college credit courses a total of $80 million each 
year and the program was suspended. There was much criticism that “teachers went and took basket 
weaving” and then were entitled to receive between $1,100 and $3,300 a year as long as they 
continued teaching. The suspension applied only for any new participants while previous participants 
continue receiving their supplements until they retire. The fiscal year 1997-98 cost to fund the 
program was approximately $38 million for professional development teachers received in the early 
1980s. 

Also in the late 1980s, Louisiana passed a state-level teacher evaluation program. 
Certification could be revoked based on the evaluation. This program soon ran into multiple 
problems and road-blocks and was totally revamped in 1992. Today, there is a state-level 
assessment for new beginning teachers tied to initial state certification. This program was revised 
in 1997 to focus more on the mentoring aspects of assisting beginning teachers. Individual 
evaluations for all certified teachers and principals are conducted at the local level based on state 
established program standards. Again, as with the curriculum and testing, the teaching standards and 
certification was the responsibility of one bureau and office while the teacher assessment and 
evaluation were the responsibility of a different bureau and office within the LDE. 
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The same fragmentation and incoherence existed for the school standards for approval 
(legislation passed in 1975) and the “School Report Card” (legislation passed in 1988). The data 
published on the “School Report Card” had very little to do the school’s annual approval rating. 
Again, these programs were administered by separate bureaus in separate offices of the LDE. 
Educational Reforms. 1992-1997 

In 1992, the teacher evaluation program was revamped and focused on state-level teacher 
assessments for beginning teachers. Local evaluations for non-beginning teachers was strengthened, 
but was left at the local level with the state only monitoring the local process. One of the major 
reforms of 1992 was the passage of a new state funding formula. This is considered major, because 
Louisiana previously, since 1929, had used a specific resource-driven instructional unit formula for 
state aid equalization and distribution. The 1992 formula was the first step in moving to a 
performance-based management system with a per pupil driven formula. The formula also made 
tremendous improvements in equalization. The previous formula had only considered $60 million 
in local revenue as the equalization factor. The new formula considered $ 1 billion in local revenue 
as the equalization factor. The correlation to local wealth of the state aid distribution went from an 
inverse relationship of .18 in 1991-92 to an inverse relationship of .87 in 1 996-97. 1 

In Louisiana, major state reforms usually come the first legislative session of a Governor’s 
first or fifth year in office. At least, that was before the legislature and the people passed a 
constitutional amendment in 1992. This amendment provides consideration for only fiscal issues 
in even numbered years and only non-fiscal issues in odd numbered years. Our laws also provide 
that Gubernatorial and Legislative elections occur in odd numbered years with the induction into 
office in January of an even numbered year. Needless to say, newly elected governors and 
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legislators become very frustrated their first year in office when they can only deal with education 
and other non-fiscal issues in a special legislative session. 

As a result, in 1996, we had the first year of a new administration and a new legislature that 
could not consider or pass any education legislation without it being in a special session! Therefore, 
there were over 1,100 education-related bills filed in 1997. 

Reforms in 1997 include a major reorganization of the Department of Education and it’s 
budget structure, a performance-based school accountability program, funding for development and 
implementation of a new testing program based on challenging content standards, a strengthening 
of the new teacher mentor program, a leadership program for principals, funding for classroom-based 
technology, funding for teacher classroom instructional supplies, funding for a K-3 reading and math 
reform program, and the expansion of the charter school program and loan fund. 

Integrated Public Policies 

Systemic reform implies policy coherence. The general notion is that policies should 
integrate and send consistent signals. This aspect of reform reflects significant frustration because 
of the complexity of the policies used to govern elementary and secondary education. There are a 
number of reasons for the complexity such as: our multilevel, multi factor system for governing 
education; the need to claim electoral credit through initiatives that are deliberately crafted to be 
distinctive; the increasing volume of policy from all levels of government; and related trends to 
specialize policy institutions. For these reasons along with many others, policy has been marked by 
fragmentation. 2 (Fuhrman, 1994) To improve the level of policy coherence, states are beginning to 
use their emerging student outcome standards as anchors for other policies that would coordinate 
with and reinforce these standards. Key policies being integrated include those directly linked to 

DRAFT, April 7, 1998, Page 6 



O 

ERLC 



57 



curriculum: instructional materials, student assessment, teacher certification and teacher professional 
development. 3 (Smith and O’Day 1991) 

In Louisiana, the 1997 reforms are focused on the challenging new content standards that 
have been adopted for students and the components of effective teaching identified for teachers. The 
reform efforts, such as new student assessments, textbook adoption, school effectiveness and 
assistance, school accountability, and leadership efforts, are being aligned with both the challenging 
content standards and the components of effective teaching. In addition, the Department 
reorganization aligned and focused the functions to be provided by a state department for students, 
schools, educators and school systems. The aligned functions are: setting standards; performing 
assessments; ensuring accountability; providing assistance; and resource management. 
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KEEPING REFORM ALIVE IN THE PUBLIC POLICY ARENA 



One of the first lessons to learn in managing public policy is that there are always three 
aspects to every issue. You may have heard of the good, the bad and the ugly? Well, the three 
aspects could be considered similar but they are called the philosophical, the technical, and the 
political. The best way to define the three areas may be by describing the types of personalities that 
generally tend to make up the resident expertise in each of the areas. 

Philosophical 

The philosophical is best typified by the university professor. University professors are very 
versed in the philosophical and research basis of almost every educational issue and can be an 
invaluable resource. However, sometimes, they have problems if they are charged with the technical 
implementation of a reform effort. Their approach is generally too “pure” or dictatorial with little 
accommodation of different perspectives. 

Technical 

The technical area is best typified by the public financial administrator. They are very good 
at figuring out the most obscure detail of very complex formulas and procedures because, after all, 
everything must balance back to the penny! An example would be the one person in the state who 
knows every detail of the state funding formula. Almost every state has the one person (or at a 
maximum no more than two or three) that is the financial administrator over billions of dollars and 
administers a program that very few people understand. These types are usually very detail oriented, 
are reluctant to change, and think in terms of numbers! 

Political 

The politician has a chameleon personality, and justifiably so. They are elected to represent 
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constituents. Therefore, their perspective shifts or changes given the changes in the perceptions or 
opinions of their constituents. The perceptions or opinions of constituents are often formed by the 
philosophical or technical applications of public policy. But don’t ever underestimate that almost 
every citizen has had some experience with public schools either directly or indirectly, usually have 
very specific opinions on how schools should operate and can become very vocal about expressing 
their concerns to elected representatives. 

Louisiana Experience 

An example of the failure to align all three aspects in Louisiana is the state level teacher 
evaluation program of 1988. From the philosophical perspective, it was very sound. It was designed 
by a team experts from a university. It ran into tremendous problems in the technical design and 
implementation stage, which had been contracted out to the University. Needless to say, with the 
technical design and implementation being very problematic, it soon became very political. We 
even had a Governor, on election night when it became clear that he had won, say “...run out of the 
state...and ... take his bastard step-children LTIP and LTEP with him...” When this happens, reform 
efforts, no matter how philosophically sound, are either suspended or required to undergo a massive 
redesign. Fortunately, in this Louisiana reform effort, a redesign was proposed and politically 
accepted so that the $5 million invested in the initial design and implementation was not a total write 
off. Unfortunately, reform efforts that go astray and are ultimately suspended usually cost millions 
of dollars for which there is little or no gain and generate significant mistrust with the public. 

What does all of this mean for encouraging change in public policy to support school 
improvement? It means any new public policy needs to be grounded in a research-based philosophy 
that is understandable or logical, can be technically implemented without undue burden or 
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discrimination and is supported by a majority group of constituents. If any of these aspects are 
missing or are not aligned with the effort, the reform may be in jeopardy of achieving it’s goal. 
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EDUCATION REFORM ~ INCLUDING ALL OF THE PIECES 



Successful education reformers develop practical strategies to manage change in a systematic 
way. School reform can be a complex undertaking that requires careful thought and administration, 
and nearly all reforms, regardless of their scope or intended target, share a number of characteristics. 
The U. S. Department of Education first published a report “Fitting the Pieces, Studies of Education 
Reform” in October 1996 which reviewed and described the essential elements of planning, 
implementing, and sustaining school reform. The report was designed to assist policy makers and 
practitioners at the district, school and community levels in creating strategies that will enable them 
to increase student learning. The essential elements as presented in the report are provided in this 
section with a comment about Louisiana’s latest reform efforts. 4 (Klein, Medrich, Perez-Ferreiro & 
MPR Associates, Inc.) 

Planning 



Leadership. School reformers usually start out with high levels of energy and 
commitment to the process. Unfortunately institutionalizing change is a long, 
arduous process, and over time competing responsibilities and the slow pace at 
which gains are made can drain enthusiasm. Strong leadership can help ensure that 
initial reform objectives are achieved. In addition to making key decisions and 
following through on their implementation, effective leaders build consensus, 
promote buy-in, and delegate authority among participants. Ultimately, strong 
leadership is about good management. 4 (Klein, Medrich, Perez-Ferreiro & MPR 
Associates, Inc., p. 5) 

The current State Superintendent Cecil J. Picard brings to the job twenty years of experience 
as a teacher, coach and principal plus an additional twenty years of experience as a state 
representative and state senator. Superintendent Picard has been actively involved with the Southern 
Region Education Board in addition to handling much of the state’s education reform legislation 
over the past twenty years. His leadership has already been demonstrated with the reforms that have 
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been initiated or are being developed since he took office on July 1, 1996. Also, since 1996 there 
has been a climate in Louisiana of the teachers, principals, school boards, SBESE, the state 
legislature and the governor all working together to improve teaching and learning that has not been 
present in past reform efforts. 

Goals. Education reforms endure when key stakeholders internalize reform goals 
and support implementation strategies. Regardless of the type of reform, positive 
results are more likely when everyone supports the larger vision, understands their 
role in the process and takes responsibility for action. Reformers seeking to create 
an environment that supports change must engage in a variety of planning and 
consensus building activities that motivate individuals to join the effort / (Klein, 
Medrich, Perez-Ferreiro & MPR Associates, Inc., p. 9) 

The school and district accountability reform underway in Louisiana has been developed by 
a Commission of representatives of the major stakeholders with a clear vision of improving student 
academic achievement. In addition, the State Superintendent has personally visited over one-third 
of the school districts in the state and spent time in the schools reviewing local reform efforts and 
seeing first-hand the successes schools and districts are achieving. These visits usually include not 
only a visit with school district and school personnel, but in many cases, the Superintendent also 
speaks with the local Rotary Chapter or the local business representatives. These local visits 
generally receive press coverage in the local communities informing a broad range of individuals 
of coming educational reforms. 

Timing. Reform is neither easy nor quick; many reforms require years of work 
before producing measurable results. While patience and perseverance are two of 
the strongest assets of any education reformer, there are a number of strategies that 
educators have developed to increase their likelihood of success. These include 
making the most efficient use of available time, and capitalizing on the opportunities 
that risk may provide A (Klein, Medrich, Perez-Ferreiro & MPR Associates, Inc., p. 

13) 
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Louisiana’s current reforms began and have been gradually building speed since 1992. As 
such, the first baseline year for school performance is 1998-99 for elementary grades K-8 and 2000- 
2001 for grades 9-12 with the first performance cycle being Spring, 2001 and Spring, 2003, 
respectively. Also, the Accountability Commission has recommended both ten year and twenty year 
growth targets with incremental progress expected every two years as a process to nurture reform 
efforts. 

Implementing 

Training . Participants must have training before they implement reforms. Education 
reform often requires fundamental changes in individuals ’ roles and responsibilities. 

This may mean that stakeholders are asked to assume new tasks in addition to their 
normal day-to-day duties, or engage in activities that have little in common with 
their present work. Regardless of the scope of change, all individuals must have the 
skills that will enable them to support reform objectives. Ideally, this training occurs 
before program implementation so that individuals have time to understand and 
relate their new skills to reform objectives, and to identify confusing issues and 
potential sources of conflict. ” 4 (Klein, Medrich, Perez-Ferreiro & MPR Associates, 

Inc., p. 15) 

Another of the activities that has significantly impacted the LDE staff and local school 
system staff in the planning and consensus building stage has been the School Effectiveness and 
Assistance Pilot (SEAP). For LDE staff to participate, they initially receive a two day training 
session on local assessment instruments to be used and the State’s components of effective teaching. 
Almost one-third of LDE staff have been involved with SEAP over the past two years. The initial 
pilot involved four school districts and 12 schools in the first year of 1996-97 and 45 schools in 23 
districts the second year of 1997-98. These numbers represent approximately 38% of Louisiana 
school districts being involved with these pilots. The change in perceptions of both LDE staff and 
local school system staff by bringing the reforms into their immediate world has also provided an 
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opportunity for them to contribute in the reform process. In the past, reform efforts have been 

developed at the top and then forced down to the classrooms with very little training involved. With 

this pilot, training along with input on the development of the reforms is being provided from the 

classroom up. Hopefully, this process will create a greater buy-in from teachers, principles and local 

school district staff in addition to the training they have received as well. 

Flexibility. Reform strategies should be flexible to accommodate multiple solutions 
to a given problem. It is often difficult to reverse direction once a reform strategy 
has been selected. Shifts in political climate or changes in fiscal resources, however, 
may require a change in reform plans. Alternatively, unexpected results of a reform 
may suggest other, equally promising paths to achieving intended goals. Successful 
reformers are agile: they adopt strategies that can accommodate a range of 
approaches and that respond to changing conditions and needs. This includes 
planning multiple solutions to a given problem, developing realistic time lines that 
avoid “all-or-nothing” approaches, and anticipating conditions that may contribute 
to achieving reform goals. 4 (Klein, Medrich, Perez-Ferreiro & MPR Associates, Inc., 

P-19) 

The primary aspect of SEAP in years one and two has been the development of an 

assessment model that local school system staff can be trained on to perform themselves without 

state involvement. The LDE has tried to maintain the approach that the on-site visits of SEAP were 

technical assistance oriented and that the use of the assessment information in revising or developing 

the individual school’s improvement plan was strictly a local decision. The LDE has maintained the 

approach that local school systems should and could best identify potential solutions to solve their 

problems. The state’s role is one of assistance and leadership in identifying “models of excellence” 

and “best practices” for schools demonstrating weaknesses. 

Infrastructure. Reform may require redesigning organizational infrastructure. 
Education reformers look for the most effective ways to organize and deliver 
classroom instruction and school services. Regardless of the depth and breadth of 
an undertaking, by definition reform involves change. At some point in time, those 
involved in the reform process are asked to break with convention and do things in 
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new ways. Invariably, changing the way things are done can affect organizational 
infrastructure. Mechanisms to disseminate information about a reform and its 
potential impact can reduce anxiety about a proposed change and support a dialogue 
that increases cooperation among participants. 4 (Klein, Medrich, Perez-Ferreiro 
& MPR Associates, Inc., p. 23) 

The reorganization of the LDE is an integral component of the current educational reforms. 
The LDE had gradually over the past thirty years become organized around federal funding sources. 
There was an Office of Special Education, an Office of Vocational Education, a Bureau of Title 1, 
a Bureau of Adult Education, and other units designated by major funding source. The 
reorganization which is being implemented during the 1997-98 fiscal year, reorganized the LDE by 
function. For the first time, there is an organizational unit for School Accountability, Improvement 
and Assistance which has mixed funding from state general funds, federal elementary and secondary, 
special education and vocational funding, and statutory dedicated funding all directed a single vision 
and goal. Previously, these funds would have been split into several different organizational units, 
each operating under their own guidelines with separate visions and goals. The reorganization was 
the initial attempt to breakdown walls established by various funding sources and to create a process 
of education guided by five offices. Those offices are: the Executive Office of the Superintendent 
for leadership; the Office of Management and Finance for resource management; the Office of 
Student and School Performance for student and school standards, assessments, accountability and 
assistance; the Office of Quality Educators for standards, assessments, accountability and assistance 
for teachers and principals; and, the Office of School and Community Support for the standards, 
assessments, accountability and assistance in non-academic school support functions, such as 
transportation, food service and parental and community involvement; and in post-secondary adult 
education and training including workforce development programs. 
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Sustaining 



Managing Resources. Reform prospects improve if there is a means to redirect or 
reallocate resources in ways that meet the needs of the new, emerging system. 

Reform is not cost-free. At the very least, reform often requires reallocating existing 
fiscal and human resources. Authority over budgets and other resources allows 
schools to support the reform process, often because it enables those implementing 
the reform to funnel resources to where they are most needed. Many schools that 
were successful at reform did not wait to be granted control over existing resources, 
but actively sought new funding. Beyond initiating the reform process, schools need 
to take into account the ongoing costs of reform in order to plan for the long run. 4 
(Klein, Medrich, Perez-Ferreiro & MPR Associates, Inc., p. 27) 

The next stage of SEAP that will be developed in 1998-99 is the school improvement plan 

process that incorporates all funding sources. One of the problems with previous reform efforts has 

been that activity was focused by using new funding that was provided specifically for that purpose. 

When the incremental reform funding went away, so did the reform effort, without ever impacting 

the underlying base. In Louisiana we are trying to model reform practices that local school systems 

can use to improve student achievement and school performance through modifying current 

management practices and spending priorities. Through this process, if whatever amount of reform 

funding is decreased or discontinued, the benefits of the reform, in terms of changed practices, will 

continue. This empowerment and training of local school system staff will also provide them the 

flexibility and tools to identify and solve their own problems. 

Self-assessment Reform is an ongoing process. School reform is a dynamic process 
that requires participants to engage in nearly continuous self-reflection and program 
improvement. Although a carefully planned reform strategy may provide initial 
support in achieving program goals, most reforms proceed by trial and error, which 
means that activities require constant review and adjustment. Maintaining 
educational gains may also require constant skill upgrading and professional 
development among reform participants, while shifts in fiscal, material, and human 
resources as well as changes in school and community demographics, can influence 
reform strategy. 4 (Klein, Medrich, Perez-Ferreiro & MPR Associates, Inc., p. 31) 
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One of the strategies in Louisiana is to create an information technology system that allows 
for the aggregation and measurement of performance indicators at all levels of the educational 
system: student, teacher, school, school system and state. It is hoped that through the use of a 
decision support system, which makes information more accessible, policy-makers and management 
officials will appropriately engage in self-reflection and program improvement. 
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CURRENT AND FUTURE CHALLENGES 



Louisiana is constantly in an environment that requires tremendous vigilance to nurture and 
protect reforms. With a fairly high percentage of students in non-public schools (approximately 
1 7%) there is always an appearance of “competition” between public and private schools. The 
demographics of the state play a large role in predicting the choices and outcomes of schooling along 
with the current and future challenges facing the state. 

Equity and Adequacy of Funding 

As has been seen from the state’s efforts to equalize funding, the most wealthy school 
systems in Louisiana only spend an amount per student that is equal to the average in the southeast 
region of states. Recent efforts to equalize state funding have achieved a certain measure of success 
when correlating the distribution of state aid to the wealth of the local district. However, Louisiana, 
like other states, is struggling with the issues of identifying adequate funding based on performance 
given the complexities comprising the nature of each of the school systems and their local 
management decisions. 

Major reform occurred in Louisiana’s school finance system in 1992. The state had 
traditionally used a minimum foundation approach to state aid funding. This system was based on 
an allotment of a teacher instructional unit based on school membership, with extra teachers and 
school administrators pyramiding from the base. These allotments were not funded unless a local 
school system also “hired” a person to fill the allotted spot. Additional categorical line items were 
funded for textbooks, vocational programs, special education, etc. In the 1992-93 year, this 
foundation type program was replaced with a percentage equalizing formula. The state formula no 
longer provided categorical line item funding. For a state with a very strong centralized approach 
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to education, the inability to “micro” manage from the state level has been one of the most difficult 
reforms to sustain. For the past two years, lawmakers have bypassed the per pupil formula to 
provide teachers a state pay raise, regardless of significant inequities in teacher pay across the state. 
The state pay raises are distributed evenly to all teachers, regardless of local wealth. Even though 
we are close to putting in place school performance accountability, state law makers appear 
unwilling to forgo traditional management practices in favor of performance management practices. 
Blending Traditional and Performance Management - A Research Model for the Future 

One of the strategies being developed in Louisiana is analysis of information that will clearly 
identify the level and use of available funding to achieve acceptable student performance levels. The 
SEAP deals with one side of the cost-benefit equation. Through the new accountability system being 
developed, Louisiana will be identifying and differentiating the performance levels of all schools. 
By merging the information system that collects data on school system staff with the annual financial 
reporting system, Louisiana is able to identify an estimate of the cost of providing educational 
services at each individual school. Current plans are to define a research project for 1998-99 that 
looks at the cost-effectiveness of Louisiana schools to determine an estimate of what an adequate 
education in Louisiana costs. It is hoped that through research-based statistics, trust between state 
law makers and local school systems can begin to grow to the point of balancing the traditional 
management approach and control of funding with the newer performance management strategies 
and flexibility of funding. 
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Introduction 



ACT 478 of the Regular Session of the Louisiana Legislature required the State 
Board of Elementary and Secondary (BESE) to develop and implement a statewide 
system of accountability for schools and school districts in Louisiana based on student 
achievement and minimum standards. The legislation specified that the system would 

(a) set ‘clear and appropriate’ performance standards for schools and school districts, 

(b) provide for technical assistance to schools and school districts deemed in need, 
and (c) include rewards and corrective actions based on state-level assessments. The 
legislation also provided for the creation of a School and District Accountability Advisory 
Commission. The Commission was to be composed of parents, educators, etc, and it 
would make recommendations to BESE regarding the structure of the accountability 
system. Finally, the bill called for BESE and the Louisiana Department of Education 
(LDE) to provide for a pilot study which would provide useful information for the 
accountability system. 

The School Effectiveness and Assistance Project (SEAP) is a joint effort 
between the LDE and researchers at Louisiana State University. It was designed to 
build LDE capacity to (a) identify schools in need,(b) analyze their needs, and (c) 
provide assistance in improving. Accordingly, the project was divided into three phases: 
Phase I focused on the design and piloting of a statewide school effectiveness model; 
Phase II focused on the design and piloting of a school analysis/diagnosis process; and 
Phase III focused on the development and piloting of a school improvement process. 
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This document presents results of initial efforts in Louisiana to develop a school 
effectiveness model. Both the model developed as part of SEAP-I and the model 
proposed by the Accountability Advisory Commission are discussed. Standardized test 
data from the Louisiana Statewide Assessment Program for the 1996-1997 school year 
are used to investigate the outcomes of the two proposed models. Also, due to their 
growing popularity among researchers, a multilevel modeling strategy for identifying 
effective schools is also studied. 

This study was designed to provide answers to two questions: 

1 . To what extent do the various procedures yield similar rankings of schools with 
regard to effectiveness status? 

2. What are the pros and cons of various procedures with regards to (a) technical 
issues, (b) practical implementation issues, and (c) political issues? 

This manuscript is organized as follows: Part 1 presents an overview of school 
effectiveness research related to the identification of effective and ineffective schools. 
Part 2 describes the three models studied. Part 3 describes the data sources and 
measures, and part four presents the results of the study. 
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Background 

Over the past twenty years there has been considerable effort focused on the 
identification of schools that appear to be unusually effective at fostering of inhibiting 
desired student outcomes such as achievement, attendance, etc. (Raudenbush & Bryk, 
1986). These efforts have focused on both conceptual issues and methodological 
issues. 

Conceptual Issues 

On what basis is it reasonable to conclude that one school is more effective than 
another at fostering student achievement or some other desired student outcome? Is it 
sufficient to simply compute building level averages of student level data and proceed 
with comparisons? The present day consensus among researchers seems to be a 
resounding "no” to this strategy. Instead, researchers have proposed a variety of 
techniques that attempt to address the fact that schools must serve student populations 
that differ significantly in their preparation for learning, out-of-school support systems 
for academic pursuits, etc. These and other exogenous variables which impact student 
outcomes are largely beyond the control of school officials and yet, many have 
concluded that they ought be taken into account in efforts to identify the unique impact 
a given school may have on student outcomes (e.g., see Goldstein, 1991). 

Raudenbush and Willms (1995) present a structural model of student 
achievement which serves as a useful platform for a discussion of the complexities of 
defining and estimating school effects. Accordingly, 



Student Achievement = Mean Achievement + Student Background 

+ School Context + School Process 
+ Random Error 

According to this model, student achievement is a function of student background, 

(e.g., social class, previous achievement, etc.), school processes (e.g., organization 
and management practices, school climate, etc.), school context (e.g., size of student 
body, percent minority, etc.) and a component due to random chance. In this 
formulation, the unique impact of a school on students is conceptualized as the impact 
beyond the influence of student background factors. Raudenbush and Willms 
distinguish two types of school effects. Type A effects reflect the overall impact, both 
context and process influences, of attendance at a given school and Type B effects 
reflect the impact of a school on student performance that is attributable to school 
practices. Type A effects are likely of interest to parents that simply want the greatest 
boost to their child's performance irrespective of whether it stems from school context 
or process. 

In this paper we focus on the estimation of Type A effects. According to 
Raudenbush and Willms, to achieve this it is only necessary to control for student 
background factors when estimating the unique impact of a school. 

Methodological Issues 

Almost 30 years ago Dyer, Linn, & Patton (1969) recommended that conclusions 
regarding school effects on student learning be based on the 'discrepancy' between 
actual mean achievement and the achievement expected given previous achievement 



and the social background of the students. As implied, the proposed strategy was 
simply to aggregate all student-level data to the school-level and, using multiple 
regression, calculate a standardized discrepancy score between actual student 
achievement and predicted student achievement. Criticisms of this approach have 
included (1) parameter estimation is generally inefficient, (2) problems of collinearity are 
common, (3) school effectiveness indices based on this model are generally unstable, 
and (4) the focus on means may mask important differences among subgroups of 
students with regard to school effects (Aitkin & Longford, 1986). 

As an alternative to the school aggregate model, a number of researchers have 
studied student-level residual models. In this instance, students from all schools are 
pooled together and, without regard for school membership, the criterion of interest is 
regressed on student background and other student variables considered relevant. The 
residuals from this total sample regression are then averaged by school and taken as 
an index of school effectiveness. However, it can be shown that this strategy will yield 
biased estimates of school effects if there is heterogeneity of slopes among schools 
(Aitkin & Longford, 1986). 

Finally, in response to the inherent multilevel nature of schools (students nested 
within classes; classes nested within schools, etc.) many researchers argue in favor of 
so-called hierarchical or multilevel statistical models as suitable for estimation of school 
effects. These models attempt to explicitly incorporate the layered nature of 
contemporary schooling into the analysis process and they permit the researcher to 
explicityly model complex within-school processes as consequences of between-school 
characteristics (see Mason, Wong, & Entwisle, 1983; Raudenbush & Bryk, 1986; 



Goldstein, 1986; Aitkin & Longford, 1986). 

Following the notation of Raudenbush and Bryk (1986), let Yy be the outcome 
(e.g., achievement score) of student i 0=1,...,^) in school j (j=1,...,J). The within school 
regression model for predicting Y from, say, social class (X), is given by, 

Y, = Pj„ + PjiX,, + e, (1) 

where (B j0 is the intercept of the regression, (3 n is the slope and ey is an error term. The 
model in Equation 1 allows for the possibility that the regression parameters (3 j0 and (3 n 
vary across schools. If this variability is manifest, it could be due to differences in 
school policies, practices, or organization. A second model, the between model, might 
attempt to predict this variability on the basis of these or other school level attributes. If 
P represents an attribute of interest, then, 

Pjk = $0k + $ik p jk + a jk ( 2 ) 

where k=0, 1 for the model in Equation 2. If Equation 2 is substituted into Equation 1 
the result is, 

Yy = $00 + $10 P j0 + $01 X ij + OliPjiX, 

+ (ey + a j0 + a n x y ) (3) 

This is a general model for multilevel data problem and many researchers argue 
that it represents a significant advance in the effort to study schools and their effects on 
students. 

Applications in School Effects Research 

The multilevel model, as described above, has several desirable features: (1) it 
is possible to decompose any relationship into within- and between-group components; 



(2) multivariate between-group formulations can be studied; (3) multiple within-group 
slopes can be handled in a single analysis; and (4) slope estimates are relatively robust 
with respect to data quality. These characteristics have motivated researchers to 
extend and employ the HLM methodology in a variety of education related contexts 
(see Raudenbush & Willms, 1991). In this paper concern is with the identification of 
unusual schools. 

As noted previously, the typical strategy for identifying unusual schools has 
involved the analysis of residuals from a student-level or school-level regression 
wherein current achievement was regressed onto previous achievement and/or some 
index of social class (Dyer, Linn, & Patton, 1969). The limitations of this strategy have 
lead many investigators to consider the usefulness of multilevel models for this purpose 
(e.g., see Tate, 1988) . Specifically, referring to Equation 2, the error terms a j0 and a jt 
represent, respectively, the deviation of school j from the overall intercept and the 
overall slope of schools in the population. Once substituted into the student level model 
as in Equation 3, the sum of a j0 and a j1 Xy, which will be referred to as n y , represents the 
difference between an actual student outcome in school j and the prediction of that 
outcome based on the school level variable P. The variable TTy can then be interpreted 
as an index of school effectiveness for student i which, given the presence of Xy, may 
vary among students within a given school. 

Multilevel models have been criticized because of their complexity and because 
when used to estimate school effects, they make it difficult for small schools to appear 
effective (de Leeuw & Kreft, 1995; Morris, 1995). Also, there is evidence that, when the 
focus is on mean outcomes, the results from multilevel analyses and the school 
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aggregate model do not differ substantially (Kennedy, Teddlie, & Stringfield, 1991). 



School Acountabilitv in Louisiana 

Over the past few years several techniques for identifying effective and 
ineffective schools have been proposed or studied in Louisiana. Initial efforts 
sponsored by the LDE grew out of legislative requirements to develop academic 
performance measures of schools relative to public funding of education in the state. 
While SEAP-I grew out of LDE desires to improve upon the initial model (see Oescher, 
Black, Gunning, & Brooks, 1996; Teddlie & Kennedy, 1996), public and political 
responses to the early effort informed the design of SEAP-I. Specifically, the results of 
early efforts pointed to need for any school identification system in Louisiana to exist 
within the following parameters; 

Simplicity: 

1 . The logic of the system should be easily explainable to lay audiences. 

2. The number of factors/variables included in the system should be limited 
and should be linked to school rankings in a clear and obvious way. 

3. The variables or factors inluded in the system should be of a high quality 
and should not be easily manipulated. 

Fairness: 

1 . The system should take into consideration the fact that some schools and 
school systems serve more or less advantaged populations than others. 

2. The system should not focus exclusively on relative rankings, but should 
incorporate absolute measures of performance (e.g., average 



achievement). 

In this section of the paper we describe the three models studied and discuss each in 
reference to these lessons learned from previous efforts. 

The SEAP-I Model 

Analyses of standardized achievement data for public school students in 
Louisiana over the past five years have shown that five predictors yield satisfactory 
results for the school aggregate regression model. These predictors are (1) percent of 
students receiving free school lunches, (2) percent of students identified as special 
education, (3) percent of students identified as gifted and talented, (4) percent of 
students classified as 'Limited English Proficient, and (5) community type or degree of 
urbanicity. 

These predictors have been used to predict school averages on a composite 
formulated with norm- and criterion-referenced standardized test data from the 
Louisiana statewide student assessment program (Crone, Lang, Franklin, & Halbrook, 
1994; Kochan, Tashakkori, & Teddlie, 1996). Analyses conducted within school 
category-- Elementary, Middle, High, Combination— have consistently yielded squared 
multiple correlations in the .6 and above range (see Teddlie, 1996). 

Typically, the school aggregate regression models studied in Louisiana were 
fitted for three years of data and the effectiveness status of schools was based on the 
consistency of their standardized discrepancy, or relative performance indicators 
(RPIs), over this period. Schools with consistently large positive RPIs were designated 
as effective relative to other schools serving similar student populations; those with 
consistently large negative values were designated as ineffective, and those with values 



consistently close to zero were designated as average. Other patterns were designated 
as inconsistent. It should be noted that the use of multiple years of data for purposes of 
identification of effectiveness status addresses one of the key challenges to the school 
aggregate regression model, instability of rankings. 

The SEAP-I study foucsed on the exploration of this model with data collected 
over the period 1994-95, 1995-96, and 1996-97. Additionally, as part of the SEAP-I 
strategy, an absolute measure of school performance, called a baseline performance 
indicator (BPI) was included in the identification process. Specifically, because 
Louisiana uses a criterion-referenced statewide test in several grades, the percent of 
students attaining the performance standard for any grade and any subject area was 
computed for each school. This value was taken as a measure of absolute 
performance. A two by two matrix (relative by absolute indicators) of school 
performance was formulated. Classifications of schools was then based on both RPIs 
and BPIs. As an example, if a .2 RPI (plus or minus) was selected as a differentiating 
point, and 85% was set as a differentiating point for the BPI, the matrix would be as 
follows: 





RPK-.2 


-,2<=RPI<=+.2 


RPI>+.2 


BPI >=85% 


Class 1 


Class 2 


Class 3 


BPI <85% 


Class 4 


Class 5 


Class 6 



In this example, schools in ‘Class 4' would be designated as ‘ineffective’ and those in 
‘Class 3' would be designated as ‘effective.’ While there is some evidence to suggest 



that selection of the boundary values of the RPI can be guided by statistical 
considerations (i.e., consistency of school classifications; see Lang, 1991), the cut point 
for this measure as well as the BPI would clearly be influenced by budget 
considerations, political realities, etc. 

As described, the model studied in SEAP-I addressed many of the issues noted 
above: The model achieves simplicity in that school level data are used to predict 
performance on an aggregate measure of student achievement. The measures used in 
the model are well know to educators and are generally thought to be of 'high' quality. 
Finally, the model incorporates both a relative and absolute measure of performance. 

The Accountability Commission's Model 

The accountability system proposed by the School and District Accountability 
Advisory Commission requires that a composite score, called a School Performance 
Score, be computed for each school in the state. The score will be based on a 
weighted combination of the following: 

Criterion Referenced Test (60%) 

Norm Reference Test (30%) 

Student Attendance (10% for grades K-6 schools; 5% for grade 
7-12) 

Student Dropout Rate (5% for grades 7-12) 



The school performance score will have a minimum value of 0 and a maximum 
value in excess of 100. Following the baseline year, each school in the state will be 
expected to achieve a score of 100 on the composite after a 10 year period. Growth 
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targets will be set for two year cycles based on distance from the 10 year goal. Growth 
is expected to be linear. Also, school performance scores will be recomputed each two 
years. Finally, the model specifies that schools performing below a critical 
'unacceptable' level will automatically qualify for corrective actions. 

The Commission's model explicitly does not take poverty into consideration. The 
logic of this decision is as follows: 

1 . The Commission felt that schools should have the same expectations for 
students regardless of their economic backgrounds; 

2. Recognizing that some schools have large populations of high-poverty 
students who face greater challenges, the Commission adopted a growth 
model so that each school would be measured against its own starting 
point; 

3. Consultants from all other states advised using poverty as a separate 
variable would set lower expectations for high-poverty students; 

4. Business representatives indicated that poverty is not a factor when hiring 
qualified applicants. High school graduates are expected to be 
knowledgeable and skilled despite their economic status. 

The Commission's model achieves the objective of simplicity in that only a limited 
number of factors are considered when setting the baseline as well as growth targets. 
Further, the achievement measures used are generally regarded highly and their use 
should be easily explainable to the lay public. However, the criteria of fairneess set 
forth above is not as readily incorporated. In particular, the model fails to consider 
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differences in student populations served and as such and may be perceived as being 
biased or ‘unfair.’ 

A Multilevel Model 

The multilevel model described above has been expaned and modified to ever 
increasing levels of complexity in an effort to more realistically address the organization 
of modern schools. These developments include longitudinal models which permit 
study of cohort and panel designs (Raudenbush, 1989), cross-classified models which 
allow for the possibility that lower level units may be nested within several higher units 
(Goldstein, 1994), and multivariate and structural equation models which permit 
modeling of multiple outcomes and allow for the presence of measurement errors in 
indicator variables (Muthen, 1994). 

The application of the simple (see Equation 1) and more sophisticated multilevel 
models to the identification of effective and ineffective schools has been discussed at 
length in the school effects literature. While most would agree that these models offer 
the possibility of more realistic modeling of school processes, their sophistication may 
be an impediment to use in the practical settings. Specifically, it seems unlikely that 
even the most simple versions of these models will be easily comprehended by lay 
persons. Add to this the shrinkage phenomenon noted above, and these models also 
have questionable ‘fairness’ appeal. Given the politically sensitive nature of high-stakes 
accountability systems, this is a serious limitation. 

Nevertheless, ease of communication alone can not be a sufficient justification 
for selecting a model for accountability purposes. However, there is evidence that 
results based on multilevel models may not differ significantly from more easily 



understood regression based procedures (see Fitz-Gibon, 1996). In the present effort, 
for comparison purposes, two simple multilevel models are studied. One is based on a 
cross-sectional study of achievement for the 1 996-1 997 school year. The other is 
based on a longitudinal design and focuses on the impact of schools on the 1994-1195 
cohort of third grade students in the state (see Bryk & Raudenbush, 1 987). The only 
control or school level variable considered for either model is percent of students in 
poverty. In both instances, empirical Bayes residuals based on school averages are 
used as RPIs. The objective was to identify which schools appeared to yield larger or 
stronger growth trajectories for their students than did other students. The only control 
or school level variable considered was percent of students in poverty. The cross- 
sectional study is concerned with identifying schools which have higher average 
achievement levels than expected given the poverty level of the students served. The 
longitudinal study is concerned with the identification of schools which appear to yield 
larger or stronger growth trajectories for the 1994-1995 third grade cohort than other 
schools. 
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Methods 

Sample and Measures 

School Aggregate Achievement . This measure is based on data from the 
Louisiana Educational Assessment Program. This program consists of a criterion- 
referenced (CRT) and norm-referenced (NRT) test administered annually to public 
school students in Louisiana. The CRT is administered in grades 3, 5, 7, 10, and 11. 
The tests administered in grades 3,5,7, and 10 measure performance in language arts 
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and mathematics, and written composition is assessed in grade 10. The grade 11 test 
measures performance in social studies and science. The NRT measures included in 
these analyses reflect performance in reading, language arts, and mathematics in 
grades 4, 6, and 8. 

The composite achievement measure was formulated for each school 
Classification-Elementary, Middle, High, Combination-by equally weighting and 
combining the scale scores in each of the subject areas for the CRT and NRT 
separately. These values were assigned a mean of 500 and a standard deviation of 
100. The final composite is based on a equally weighted composite of the CRT and 
NRT. Again, the mean is 500 and the standard deviation is 100. 

Percent Free Lunch . This measure reflects the percent of students receiving 
free school lunches as reported to the Student Information System (SIS) during the 
October 1 reporting period. 

Percent Limited English Proficient . This percent is based on the number of 
students reported as LEP on the student test answer records for the CRT. 

Community Type . This variable reflects the type of community within which 
schools are located. It is based on US Census classifications of schools by population 
of the area. There are seven classifications ranging from ‘T Large City to 7' Rural. 

Percent Special Education . This percent is based on data maintained by the 
LDE Lanser data base. It reflects the percent of students falling within special 
education ‘exceptionalities’ using LDE categories. 

Percent of Gifted and Talented Students . This measure is based on data 
reported in the Lanser data base. It reflects the percent of students falling within the 



special education exceptionalities ‘gifted’ or ‘talented.’ 

The Regression Model 

The strategy for fitting the regression models reported in this paper involved (a) 
checking assumptions of the fixed predictor regression model, (b) identifying unusual or 
influential observations, and (c) building best-fitting regression functions (see Neter, 
Kutner, Nachtsheim, and Wasserman, 1996). All analyses were conducted within 
school type classifications (noted above). The resulting models yielded squared 
multiple correlations in excess of .6, with the exception of the combination schools 
which was lower. The relative performance indicators, RPIs, reported for this model are 
essentially studentized residuals (see Neter et al. , 1996). 

The Accountability Commission's Model 

This model called for the development of a composite score based on CRT, 

NRT, student attendance, and student dropout data. For the present analyses, only 
achievement data were used. The Commission's recommendations are based on the 
Louisiana Educational Assessment Program as it will exist during the 1998-1999 school 
year. Significant changes will occur not only in the grades assessed, but in the 
assessments used. Specifically, the current norm-referenced test, CAT5, will be 
replaced by the Iowa Test of Basic Skills. The current criterion-referenced test will be 
replaced by a version with a more performance based orientation. Further, the new 
CRT will classify students into one of five performance levels: Unsatisfactory, 
Approaching Basic, Basic, Proficient, and Advanced. Because the current CRT only 
classifies students into two categories, met standard or failed to meet standard, it was 
necessary to set category boundary values to arrive at five categories. We assumed 
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that 10% would achieve Advanced, 15% would achieve Proficient, 15% would achieve 
Approaching Basic, the percent failing to meet standard would be classified as 
Unsatisfactory, and the remaining students would be classified as Basic. The 
procedure for transforming these percentages into a single score was as follows: 

Identify the percentage of students performing in each category. Assign weights as 
follows: Advanced=2; Proficients .5; Basic=1 .0; Approaching Basic=0.5; and 
Unsatisfactory=0.0. Compute the total number of points and multiply this value by 0.60. 
For the NRT, we computed the percentage of students with stanines greater than 4. 
Schools that did not have grades in which the CRT or the NRT were given were 
eliminated from the analyses. 

The Multilevel Model 

For this model the focus was on the equally weighted composite of the CRT and 
NRT. School averages on the composite were computed for the 1 994-1 995 third grade 
cohort at three points in time: as third graders during the 1 994-1 995 school year; as 
fourth graders during the 1995-1996 school year; and as fifth graders during the 1996- 
1997 school year. These data were not matched at the student level for two reasons: 
First, it was no obvious how we cold handle students retained in grade and eliminating 
these students from the analyses was deemed unacceptable. Second, student records 
from the Louisiana testing program do not readily permit matching files across year. 

The level of accuracy was deemed unacceptable. 

The longitudinal model studied for this analysis assumed a linear growth 
trajectory over the three years. The within-school model predicted the average 
achievement as a function of time and percent free lunch was used as a between- 

18 




89 



school predictor. For comparison purposes, a school aggregate regression model was 
computed with these data for each of the three years. In each instance, average 
achievement was predicted as a function of percent free lunch. The objective of the 
comparison was to determine the extent to which schools were consistently classified 
as positive or negative by the two methods. 

Analysis Strategy 

Although the school aggregate regression model developed in SEAP-I was 
based on three years of data, the current analyses focused only on the most recent 
school year. The primary objective was to determine the consistency of rankings 
between the RPIs formulated with the multilevel and regression models, and the school 
performance score associated with the Commission’s model. Also, a key interest was 
in determining the extent of relationship between free lunch rates and school rankings 
based on the three methods. Pearson correlations are used for these purposes. 



Results 

The results in Table 1 present Pearson correlations between the effectiveness 
indices based on the regression model (REGRESS), the model proposed by the 
accountability commission (ACCTCOM), and the percent of students at each school 
receiving free school lunches (PCTFREE)--a measure of poverty. These data 
consistently show a higher relationship between free lunch rates and the effectiveness 
index proposed by the accountability commission than the regression model or the 
multilevel model. 

The results in Table 1 indicate that the Commission’s model may be particularly 
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harsh in terms of schools with large concentrations of high poverty students. In Table 
2 we investigate this further by presenting the mean and quartiles for free lunch rates 
for schools that meet and schools that do not meet the 100 cut score on the 
commission's model. In practically every instance, free lunches rates for the high 
achieving schools are consistently lower than are those for the low achieving schools. 

The final analyses considered in this paper concerns the degree of consistency 
between the classifications of elementary schools based on a longitudinal examination 
of the 1994-1995 third grade cohort. Using the school aggregate regression model, 
schools with positive residuals for each of the three years were designated as 
'consistently positive’ and those with negative residuals for this period were designated 
as consistently ‘negative.’ The results were that 63% of the 'consistently positive’ 
schools had positive residuals for their growth rate based on the multilevel model, and 
52% of the ‘consistently negative’ schools had negative residuals based on the 
multilevel model. 

Conclusions 

This study reviewed several techniques for identifying effective and ineffective 
schools for accountability purposes. The results suggest that the SEAP-I school 
aggregate regression model not only has rational appeal based on criteria set forth in 
this document, but that it yields results not substantially different from those reported for 
more sophisticated models. On the other hand, the model proposed by the School and 
District Accountability Advisory Commission appears to yield performance rankings 
related to poverty. Several issues stem from this: 
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1 . Can a model that appears to confound school effects with student background 
characteristics be successfully implemented? 

2. To what extent will school success in reaching growth targets be related to 
poverty level? These results indicate that the growth targets for high poverty 
schools will be more demanding than the growth targets for more advantaged 
settings. 

Finally, it should be noted that the results for the Commission’s model reported in this 
document are based on achievement data only and reflect assumptions that may or 
may not be consistent with results form the future testing in the state. 
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Table 1. 



Pearson Correlations Among School 



Indices & Percent Free Lunch 


ALL SCHOOLS 

PCTFREE 


REGRESS 


ACCTCOM 


PCTFREE 


1.00000 


0.04488 


-0.67305 


REGRESS 




1.00000 


0.53940 


ACCTCOM 






1.00000 


ELEMENTARY SCHOOLS 

PCTFREE 


REGRESS 


ACCTCOM 


PCTFREE 


1.00000 


0.03983 


-0.72735 


REGRESS 




1.00000 


0.55599 


ACCTCOM 






1.00000 


MIDDLE SCHOOLS 

PCTFREE 


REGRESS 


ACCTCOM 


PCTFREE 


1.00000 


0.05939 


-0.74387 


REGRESS 




1.00000 


0.48943 


ACCTCOM 






1.00000 
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Table 1 (continued) 



HIGH SCHOOLS 

PCTFREE 


REGRESS 


ACCTCOM 


PCTFREE 


1.00000 


0.11410 


-0.66002 


REGRESS 




1.00000 


0.47490 


ACCTCOM 






1.00000 


COMBINATION SCHOOLS 

PCTFREE 


REGRESS 


ACCTCOM 


PCTFREE 


1.00000 


0.01872 


-0.54552 


REGRESS 




1.00000 


0.62271 


ACCTCOM 






1.00000 
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Table 2. Quartiles and Mean Free Lunch Rates by Accountability Standard. 



SCHOOLS ABOVE THE 100 CUT SCORE 




1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


Percent Free Lunch: 


Elementary 

33.61 


Middle 

35.09 


High 

29.29 


Combination 

51.68 


Quartiles: 

Q3 


45.49 


52.74 


36.52 


79.53 


Q2 


32.54 


32.00 


29.29 


41.39 


Q1 


19.44 


13.88 


22.06 


34.11 


SCHOOLS BELOW THE 100 CUT 


SCORE 






Percent Free Lunch: 


Elementary 

62.22 


Middle 

52.30 


High 

43.86 


Combination 

46.26 


Quartiles: 

Q3 


83.01 


68.26 


62.80 


56.80 


Q2 


63.44 


53.78 


39.64 


44.32 


Q1 


43.47 


35.53 


26.82 


32.19 



Note. There were only 3 observations in the 'Above Cut Score' group for Combination 
schools. 
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Introduction 

The School Effectiveness and Assistance Pilot (SEAP), a joint research project 
between the Louisiana Department of Education (LDE) and faculty at Louisiana State 
University (LSU), was conceived in the fall of 1997 as a mechanism for developing 
and testing three components of a comprehensive, statewide education 
accountability system. Those components are: (a.) SEAP-I, involving the 
development of a regression-based model for measuring school performance; (b.) 
SEAP-II, a multi-method approach to collecting school process data through 
intensive, site-based research; and (c.) SEAP-III, which joins local school staff, LDE 
specialists, and university researchers in collaborative, school improvement efforts 
springing from the SEAP-II research. The following paper is an overview of the 
process used to gather data in the SEAP-II site visits, focusing on the methodologies 
that were employed and how the process benefits not only the schools which 
volunteered, but the LDE as well. 

Overview of the SEAP Process 

Roughly 18 months after its inception, the three components of SEAP are in 
various stages of implementation. During SEAP-I, which was conducted during the 
summer of 1997, composite relative performance indices (RPIs) of student 
achievement were calculated for all public schools in Louisiana for three 
consecutive years (1994-95 - 1996-97). Though the RPIs have been well received by 
policy makers at the local level, it is not likely that the performance model 
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recommended by the SEAP -I researchers will be adopted and implemented. The 
gubematorially-appointed panel that is charged with recommending an 
accountability model to the Louisiana Board of Elementary and Secondary 
Education (BESE) opposes the use of regression-based indices that take school intake 
characteristics into account. SEAP III, the school improvement phase, is in its 
infancy, as only one cohort of schools have gone through the complete SEAP 
process. 

SEAP-II is the only component that has run its full course. In addition, the 
SEAP-II site visits have become the centerpiece of the LDE's attempt to reinvent 
itself as an agency whose primary role is to support schools and districts in their 
efforts to improve themselves rather than function as a bureaucracy that audits 
rather than advises, documents rather than assists. 

What Is SEAP-II? 

The first round of SEAP-II visits was conducted over a four week period 
during the Spring of 1997. These site visits were originally designed to have been 
conducted in the Fall of 1997, but decision makers at the LDE were impatient to 
begin the SEAP process. It was decided during the winter of 1996-97 that Cohort 1 
visits would be completed before the end of the 1996-97 school year rather than "lose 
an entire year" by waiting until the next academic year. 

The format of the SEAP visits was conceptualized during the early months of 
1997 and field tested on a voluntary basis in 12 schools/ districts around the state (4 
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districts, 3 schools each). Based loosely on the model used in the Louisiana School 
Effectiveness Study (LSES) (Teddlie & Stringfield, 1993), the process called for five 
\ member teams from the State Department of Education and LSU to conduct two 
day site visits to each participating school. 

During each site visit, data on school climate and classroom instruction were 
collected using the following mix of quantitative and qualitative methods: surveys 
(parents, teachers, principal), interviews (principal), and focus groups (student, 
teacher). Classroom observations also were conducted, during which student 
time-on-task was assessed using the Stallings Time on Task Instrument and 
instructional methods and behaviors were documented as they relate to the 
Louisiana Components of Effective Teaching (LCET). School- and district-level 
reports were generated from the analysis of the school level data which, cited 
school /district strengths and made specific recommendations for improving areas of 
weakness. The following sections of the paper explore the methodology utilized for 
school level data collection in the Cohort 1 schools, i.e., participant observation, 
attitudinal surveys, interview/ focus groups, and consensus building. 

Participant Observation as a Research Tool 

Participant observation is a critical component of each SEAP site visit. Team 
members not only observe in classrooms but throughout the school, gathering data 
which is explicit as well as that of a more elusive nature. The main advantage of 
directly observing school activities and operations is that the team develops a better 
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understanding of the context within which the school operates (Patton, 1990). The 
two ways in which participant observation is utilized in the SEAP process is through 
the School Observation Checklist, and in classroom observations. 

School Observations 

The School Observation Checklist (see Appendix 1) is a form which allows for 
the standardization of school level data collection across the schools visited. The 
form encompasses areas such as the teachers' lounge, principal and assistant 
principal presence throughout the school, auxiliary classes (i.e., art, P.E., vocational 
classes, special education), cafeteria staff and rules for student behavior in the 
cafeteria, the custodial staff and school cleanliness, the library and its function in the 
school, the play ground, school arrival, and hallways and bulletin boards. The 
instrument is divided among the team members, so that each person specifically 
focusses on one or two areas, but also records any pertinent observations outside of 
the assigned section(s). These observations are invaluable in understanding the 
underlying school culture and how day-to-day management activities are 
undertaken. In addition, these observations are quite helpful when pulling together 
all of the data collected during the site visit into a written case study, as it is the 
context variables which make each school unique. 

Classroom Observations 

The classroom observation component of the SEAP-II site visits consists of 
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twenty four classroom observations spread across the five observers over a two-day 
visit. The team leaders seek to stratify the grades and subjects examined across the 
observers, so that a more holistic view of the type of teaching in use at the school 
can be obtained. The procedure used to standardize the observations process is a 
mix of both quantitative and qualitative data gathering methods that broadens the 
lens of the observation, especially when compared to more traditional methods of 
school level observations, which tend to focus on teacher activities and not on 
classroom context. 

The classroom observation process is threefold. The first stage is scripting. 
This consists of the observer taking continuous field notes of what is occurring in 
the class. In the second stage, while the observer is scripting, the Stallings 
Classroom Snapshot is used every 4-6 minutes to scan the class and record a time on 
task measurement in six equal intervals. The last stage involves the completion of 
the LCET, a quantitative instrument designed to evaluate teacher behaviors. The 
following sections will explore each part of the process in greater depth, and then 
show how the results of the classroom observations are used. 

Scripting. The scripting process begins as soon as the observer enters the 
classroom. The goal of this procedure is to have a running record of exactly what is 
occurring in the class. These notes consist of quotes from both teachers and 
students, descriptions of the room's physical qualities, explication of how students 
respond to the teacher and his/her teaching style, as well as the types of activities 
that go on during the observation. In addition to providing notes from which the 
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LCET will be scored, this process also adds a layer of context to the observation 
which is not present in the use of strictly quantitative observation methods 
(Schaffer, Nesselrodt, & Stringfield, 1994). 

Stallings Classroom Snapshot. The Stallings (1980) Classroom Snapshot is a 
"low inference instrument which allows for the calculation of time on task and 
levels of interactive teaching" (Schaffer, Nesselrodt, & Stringfield, 1994). After 
scripting the lesson for approximately five minutes, the observer scans the room 
and takes a count of those students who are engaged with the teacher, those who are 
working either in groups or alone (without the teacher), and those students who are 
off-task (unengaged in the classroom activity). The observer repeats the scan at five 
equal intervals of four to six minutes, continuing to take field notes when not in the 
midst of a scan. 

Table 1 gives an example of how the time on task data gathered in the 
classroom observations is aggregated at both the school and grade level. This table is 
information rich, and allows the analyst to draw conclusions about the types of 
teaching that are utilized at a specific grade level, as well as those visible across the 
school as a whole. For example, the rates of interactive teaching reported in Table 1 
are quite low in the fifth and sixth grades. This can be traced to the preponderance 
of written assignments that were seen at these grade levels as compared to the other 
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Table 1. Percentage of Time on Task (TOT): Total and Interactive, by Grade Level 



Task 


Grade 5 


Grade 6 


Grade 7 


Grade 8 


All 




N 


% 


N 


% 


N 


% 


N 


% 


N 


% 


Total Time on Task (TOT) 


4 


84.9 


5 


63.0 


6 


72.1 


7 


73.9 


2 3 


7 2.4 


Interactive Time on Task (TOT) 


4 


29.0 


5 


33.2 


6 


59.2 


7 


50.0 


2 3 


4 5.5 


Reading Aloud 


4 


1.34 


5 


.513 


6 


.556 


7 


.099 


23 


.519 


Making Assignments 


4 


4.17 


5 


0 


6 


2.78 


7 


0 


23 


1.45 


Instruction, Explanation 


4 


12.7 


5 


11.3 


6 


14.0 


7 


19.8 


23 


14.3 


Discussion, Reviewing Assignments 


4 


10.8 


5 


18.1 


6 


7.50 


7 


14.1 


23 


12.1 


Practice Drill 


4 


0 


5 


0 


6 


12.4 


7 


1.06 


23 


6.18 


Taking Test, Quiz 


4 


0 


5 


3.30 


6 


22.0 


7 


14.9 


23 


11.0 


Non-Math or Non-Reading Instruction 


4 


0 


5 


0 


6 


0 


7 


0 


23 


0 


Non-Interactive Instruction 


4 




5 




6 




7 




2 3 




Reading Silently 


4 


24.0 


5 


1.68 


6 


0 


7 


2.98 


23 


5.44 


Written Assignments 


4 


31.9 


5 


28.1 


6 


13.0 


7 


20.9 


23 


21.4 



Source: Observer-scored Classroom Observations , Spring 1997 



Louisiana Components of Effective Teaching. The LCET is a modified 
version of an instrument developed for the Louisiana Teacher Assessment 
program. The main difference between the two forms, is that the assessment 
version rates teachers behaviors on a two point scale (pass/ fail); while the SEAP 
version of the instrument uses a four-point scale to obtain a greater degree of 
variance in scoring. As established by the Louisiana Department of Education, the 
face validity of the instrument is quite high. Out of over 7000 responses from 
professional educators throughout the state, 93.9% found that the instrument 
represents a realistic expectation for teachers, and 95.5% said that they believe that 
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the framework as a whole represents a comprehensive set of expectations for 
teachers (Oescher & Brooks, 1993). 

The LCET focuses on two domains, management and instruction. Within 
the management domain are components which examine the learning 
environment, maximization of time, and management of learner behavior. 

Within the instruction domain are components which examine the teacher's ability 
to deliver instruction effectively, the teacher's presentation of appropriate content, 
the teacher's provision of opportunities for pupil involvement in the learning 
process, and how the teacher assesses pupil progress. Each component is further 
subdivided into attributes which are rated by the observer 

Each attribute is rated on a scale of one to four. A "4" indicates the teacher is 
doing an outstanding job. A "3" indicates that teacher performance can be 
improved, although current practices are clearly acceptable. A "2" indicates that 
improvement activities are required to meet standards consistently, and a "1" 
indicates that the teacher's performance is not acceptable and improvement 
activities must be undertaken immediately (SEAP Training Manual, 1997). 

Table 2 shows mean teacher LCET ratings by grade and school level for one of 
the SEAP Cohort 1 schools. A score on an attribute or component below a 2.5 signals 
a need for attention in that area. For example, on Domain III, Component A: the 
teacher delivers instruction effectively, the school as a whole rated below 2.5, as are 
the teachers at each grade level. In addition, every attribute in the component also 
scored below 2.5. Ratings this low would be a focus of the final report as justifying 
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targeted staff development in order to improve delivery of instruction throughout 
the school. 



Table 2. Mean Teacher Ratings on the LCET, By Grade Level 



Components & Attributes 


Grade 

5 


Grade 

6 


Grade 

7 


Grade 

8 


All 

Grades 


Domain II. Management, Component A 

The teacher maintains an environment conducive to 

learning. 


2.87 


2.80 


3.00 


2.71 


2.86 


II(A1). Organizes available space, materials, and/or 
equipment to facilitate learning. 


3.25 


2.80 


3.00 


2.85 


3.00 


II(A2). Promotes a positive learning climate. 


2.50 


2.80 


3.00 


2.57 


2.73 


Domain II. Management, Component B 

The teacher maximizes the amount of time 

available for instruction. 


2.75 


2.37 


2.70 


2.71 


2.61 


II(B1). Manages routines and transitions in a timely 
manner. 


3.00 


2.50 


3.00 


2.71 


2.77 


II(B2). Manages and/or adjusts time for activities. 


2.50 


2.25 


2.40 


2.71 


2.45 


Domain II. Management, Component C 

The teacher manages learner behavior to provide 

productive learning opportunities. 


2.87 


2.50 


2.90 


2.64 


2.71 


II(C1). Establishes expectations for learner behavior. 


2.75 


2.60 


3.00 


2.85 


2.78 


II(C2). Uses monitoring techniques to facilitate 
learning. 


3.00 


2.40 


2.80 


2.42 


2.65 


Domain III. Instruction, Component A 
The teacher delivers instruction effectively. 


2.29 


2.00 


2.36 


2.15 


2.21 


III(Al). Uses technique(s) which develop(s) lesson 
objective(s) 


2.25 


2.00 


2.20 


2.16 


2.19 


III(A2). Sequences lesson to promote learning. 


2.75 


2.00 


2.75 


2.33 


2.40 


III(A3). Uses available teaching material(s) to 
achieve lesson objective(s). 


2.00 


2.00 


2.00 


2.00 


2.00 


III(A4). Adjusts lesson when appropriate. 


2.00 


na 


3.00 


1.00 


2.00 


Domain III. Instruction, Component B 
The teacher presents appropriate content. 


3.00 


2.53 


2.93 


2.85 


2.83 


III(Bl). Presents content at a developmentally 
appropriate level. 


3.00 


2.40 


3.20 


2.85. 


2.86 


III(B2). Presents accurate subject matter. 


3.75 


3.20 


3.40 


3.00 


3.31 
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Components & Attributes 


Grade 

5 


Grade 

6 


Grade 

7 


Grade 

8 


All 

Grades 


III(B3). Relates relevant examples, unexpected 
situations, or current events to the content. 


2.25 


2.00 


2.20 


2.66 


2.28 


Domain III. Instruction, Component C 
The teacher provides opportunities for student 
involvement in the learning process. 


2.31 


1.90 


2.35 


2.42 


2.31 


III(Cl). Accommodates individual differences. 


1.75 


1.40 


1.80 


1.85 


1.78 


III(C2). Demonstrates ability to communicate 
effectively with students. 


3.00 


2.40 


2.80 


3.14 


2.86 


III(C3). Stimulates and encourages higher order 
thinking at the appropriate developmental levels. 


1.75 


1.60 


2.20 


2.16 


2.00 


III(C4.) Encourages student participation. 


3.00 


2.20 


2.60 


2.33 


2.52 


Domain III. Instruction, Component D 
The teacher assesses student progress. 


2.75 


2.20 


2.70 


2.42 


2.47 


III(Dl). Monitors ongoing performance of students. 


3.00 


2.40 


2.80 


2.42 


2.56 


III(D2). Provides timely feedback to students 
regarding their progress. 


2.50 


2.00 


2.60 


2.42 


2.39 



Source: Observer-scored Observations , Spring 1997 



Results of Classroom Observations 

As seen in Tables 1 and 2, the data generated from the classroom observations 
are aggregated and presented in summery form at each grade level. This is because 
the purpose of SEAP is to generate a holistic picture of the school, and not to single 
out individuals. 

There were a few methodological problems associated with the classroom 
observation components with Cohort I schools. The classroom snapshot yielded 
very high levels of total time on-task behavior in contrast to the low performance of 
many schools on the LCET. However, most schools that performed poorly on the 
LCET also had low rates of interactive teaching, in spite of high total time on task. 
The latter finding is in line with benchmark studies conducted throughout 
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Louisiana over the past decade, which show that schools with low rates of 
interactive time on task are less effective (Teddlie, 1994). It was felt that the overall 
high measures of total time on task were slightly skewed in part due to 
complications which arose in using the classroom snapshot scoring sheet. This 
form was revised for use in Cohort II schools, so that choices were streamlined. 

The scripting became a valuable tool when writing the reports that were 
disseminated to the school districts and their schools, as this qualitative data enabled 
the reprort writers to support the quantitative data with examples from actual 
observations. For example, many of the schools in Cohort I were rated quite low on 
components relating to the individualization and flexibility of instructional 
methods. Through the patterns which emerged from the scripting, it became 
evident that there was a low usage of hands-on learning techniques and high 
occurrences of teacher-centered instruction. This anecdotal evidence lent credence 
and support to the more structured LCET data, and provided the basis for which to 
begin making recommendations for school improvement. 

Attitudinal Surveys, Interviews, and Focus Groups as Research Tools 

Interviewing in research provides a way to find out what is going on in 
someone's mind or a person’s perception of a particular situation. Three methods 
of gathering individual perceptions about a specific school in the SEAP - II process 
are surveys, interviews, and focus groups. Surveys enable researchers to gather 
large amounts of data, which can be standardized across constructs and administered 
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across different groups. Interviews and focus groups provide details and 
information that cannot be uncovered through close-ended questions on surveys. 
Interview techniques allow a researcher to delve into the minds of the participants 
in a situation and to gather more in-depth information about the functioning of the 
organization. The following sections of the paper explore how attitudinal surveys, 
interviews, and focus groups are used in the SEAP-II process. 

Attitudinal Surveys 

The attitudinal surveys used in SEAP-II are based upon research by 
Brookover, W.B., Schweitzer, J.H. , Schneider, J.M., Beady, C.H., Flood, P.K., & 
Wisenbaker, J.M. (1978). The surveys were modified during the field testing phase 
of the Louisiana Effective School Study (LSES-II) to ensure clear wording and the 
elimination of redundant items (Teddlie & Stringfield, 1993). The surveys are 
worded in a parallel fashion and focus on the respondent's perceptions about 
various school areas, i.e. reputation of the school, school environment, school 
climate, organizational issues, parent/ school relationship, staff development, and 
job satisfaction. 

Surveys are administered to four different groups within the school 
constituency: the principal, the teachers, students and parents. Parent surveys were 
mailed to the school in advance and distributed in the homerooms of three classes 
at different grade levels. These surveys are collected at the end of the site visit. 
Principal and teachers surveys are distributed at the beginning of each site visit, and 
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collected at the end of the second day. Student surveys are conducted in three 
classes above the second grade level, and read to students between third and sixth 
grades. 

Table 3 presents the mean scores of responses on the attitudinal surveys for a 
Cohort 1 school, across all four constituencies. The scores are based upon a five 
point scale, in which "5" indicates the most positive response and "1" indicates the 
least favorable response. All scores, except for "General Reputation of the School" 
are based on multiple items; for example, the score for "Safe and Orderly 
Environment" is the average score for five items that measure different aspects of 
that component. "General Reputation of the School" is based on one item only. 



Table 3. Mean Attitudinal Responses of Students, Parents & School Staff, by Type of 



Respondent 



Respondent 


Student 


Teacher 


Principal 


Parent 


General Reputation of the School 


na 


3.08 


3.00 


3.10 


Climate I: Safe and Orderly 
Environment 


3.23 


4.17 


4.40 


3.22 


Climate II: Expectations 


4.10 


2.83 


3.00 


3.82 


Climate III: Academic Norms 


3.84 


3.78 


3.60 


3.65 


Climate IV: Academic Futility 


3.37 


3.86 


4.50 


3.71 


Organization I: Collaboration 


na 


3.69 


4.00 


na 


Organization II: Leadership 


na 


3.75 


4.20 


3.88 


Quality of Instruction 


3.91 


3.95 


3.90 


3.59 


Parent/School Relationship 


3.81 


2.99 


3.70 


3.66 


Staff Development 


na 


3.28 


3.60 


' ' - na 


Job Satisfaction 


na 


3.99 


3.80 


na 



Na=Not applicable. Source: Site-level Surveys of Parents, Students, and School Staff, Spring 1997 
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An advantage of administering the surveys across the four levels of 
respondents is that differences in perceptions regarding the school become apparent 
when compared across the constructs. For example, in the above table it is evident 
that the principal's and teachers' expectations for the students are much lower than 
the expectations of the parents and students. In addition, the principal and teachers 
perceive the school to be a safer place than do the parents and students. 

Interviews and Focus Groups 

According to Patton (1990), three general types of interviews are available to 
researchers. These are the informal conversational interview, the general interview 
guide approach, and the standardized open-ended interview. For the interviews 
conducted during SEAP-II site visits, the interview guide approach is utilized. The 
site teams are provided with a list of topics and questions to discuss with the 
principal and guidance counselor so that interview data gathered across sites is 
consistent. While maintaining consistency in the topics discussed in the interviews, 
the interview guide approach allows the interviewer to probe as needed to gather 
information important to the individual school context. 

In contrast to the principal and guidance counselor interviews, teacher and 
student interviews are conducted through the use of focus groups. According to 
Brown, Collins, and Duguid (1989), "groups are not just a convenient way to 
accumulate the individual knowledge of their members. They give rise 
synergistically to insights and solutions that would not come about without them 
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(p. 40)." 

Focus group interviews are considered to be highly efficient since they allow 
the researcher to gather data from a larger number of people at one time. 

Participants in focus groups also tend to provide checks and balances for each other 
that reduce problems with false or extreme views. In addition, the group dynamics 
of focus groups help to keep the group focused on the most important topics and 
issues, and allow the researcher to determine if there is a fairly consistent view of 
the program or school among the participants. 

Due to the number of participants in such group interviews, the response 
time must be increased therefore the number of questions asked may have to be 
limited. In addition, facilitating interviews with groups requires some knowledge 
of group process skills so that the discussion will not be dominated by one or two 
individuals in the group. It is generally necessary to have at least two researchers 
conducting focus group interviews since it is difficult to take notes while facilitating 
the discussion of the group (Patton, 1990). 

For the student and teacher interviews conducted during the SEAP site visits, 
a list of questions are provided for the team similar to the interview guide 
technique. The team utilizes these questions in conducting the interviews, but also 
probes as necessary to gain needed information or to clarify information based upon 
observations at the site. The interview topics include areas such as the strengths 
and weaknesses of the school, methods of improvement, job satisfaction, and future 
aspirations. Discussion of these topics by the group generally provides details that 
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verify or expand upon the data gathered through observations and attitudinal 
questionnaires. 

Results from Interviews and Focus Groups 

The information gathered in the interviews and focus groups provide 
interesting anecdotes at each of the schools that add depth to the quantitative data 
collected through the SEAP methodology. These anecdotes make the reports more 
interesting and often reveal unique circumstances in certain schools. 

In the case of one school in north Louisiana, the principal interview provided 
information about the numerous business partnerships that are held with the 
school. These partnerships yielded both resources and support for the school. The 
principal shared how the school was involved in partnerships with the local cable 
company, the local K-Mart store, a local hospital, the local Domino's Pizza, a 
Kiwanis Club in the area, and an Air Force base located nearby. 

The cable company had provided cable hookups to all of the classrooms, TV 
production equipment, and a closed circuit TV channel enabling the students to 
produce their own shows. In addition, the company had installed television sets 
purchased by the school's PTO in each classroom. These improvements allowed the 
school to produce a morning news program staged by the students. 

While this company had chosen to participate by providing resources to the 
school, others participated by providing incentives for the students. The K-Mart 
store sponsored a Citizen of the Month program and a Lunch Bunch Program, while 
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the Kiwanis Club sponsored the B.U.G. (Bringing Up Grades) roll and Terrific Kids 
programs. The local Air Force base had taken an even more hands-on approach to 
their partnership by participating in a tutoring program that provided assistance to 
individual students while providing service people at the base with an opportunity 
to fulfill their community service requirements. 

The information gained from the principal regarding these programs was 
very important in the development of the school's final report. It provided details 
regarding the school climate and incentive programs that might not have been 
observed during the three-day visit, or gleaned from the closed ended interviews. 

The teacher and student focus groups were consistently found to be valuable 
information generators across all twelve schools. For example, at one of the schools, 
the principal had portrayed himself as a strong disciplinarian during his individual 
interview. However, through the teacher and student focus groups, it became 
increasingly apparent that this was not the perception of others in the school 
community. Teachers indicated that the principal utilized a negative approach to 
discipline which disturbed many of the teachers. They felt that he seldom offered 
praise, and even remarked that after a recent program for which the students had 
worked very hard, he failed to congratulate them for a job well done. Instead, the 
principal commented that the students talked too much and too loudly in the 
hallway while waiting to perform. Considering that the students were performing 
the program at several other schools in the district and had been recognized for their 
performance by the district superintendent, the principal's behavior seemed to be a 
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rather harsh response. 

The students who participated in the focus group interview corroborated the 
teachers' tale of a negative disciplinary environment. They complained of partiality 
in the enforcement of the school rules, including a story about a group of honor roll 
students who were caught in the bathroom smoking marijuana yet went 
unpunished because of the likely damage to their grades, and to the school's 
reputation. They were also concerned about the controlling and negative comments 
of the principal, remarking that he seldom had anything positive to say. The 
students reported that the principal's comments to students tended to focus upon 
reminders to students to tuck in their shirts or get haircuts. 

Anecdotes such as the ones described above provided substance to support the 
results of the attitudinal questionnaires, the test data, and the classroom and school 
observations. In addition, they were also useful in substantiating recommendations 
made in the final reports. 

Issues Related to the Use of Interviews 

One of the major issues in utilizing focus group interviews in school settings 
such as those visited during the SEAP site visit is that of confidentiality. Whenever 
a group of individuals who work together daily are asked questions about their 
work setting as a group, differences of opinion about the situation will come out in 
the discussion. Therefore, it is very important to insure that all parties involved in 
the interviews agree to maintain the confidentiality of the group once the interview 
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is complete. 

This issue was raised early in the SEAP visits when a teacher at one of the 
first schools visited went to the principal after participating in the teacher focus 
group. This teacher repeated comments made by other teachers who participated in 
the group, and apparently provided the principal with information that he felt 
reflected negatively on him as the school's administrator. As a result of this 
information, the principal later attempted to fire the teacher who had been most 
vocal during the focus group interview removed from his position. The principal 
was ultimately unsuccessful in this attempt, but the experience triggered the 
addition of several safeguards to the data collection process to prevent a 
reoccurrence. 

The first of these safeguards is the requirement that all participants sign a 
statement of confidentiality before the focus group interview begins. In addition, at 
the beginning of the interview, the facilitator reads to the group a statement 
regarding confidentiality of all matters discussed during the interview. These 
requirements have so far prevented further confidentiality problems during the 
SEAP site visits. 

A second issue concerns focus group sampling. It was also discovered early in 
the SEAP visits that in several schools, relatives of administrators were employed by 
or enrolled in several of the schools visited. In several instances, these individuals 
were randomly selected for participation in either the teacher or student focus group 
interview. Their presence during the interview stifled other members of the group 
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in their responses, and made them very guarded. In order to eliminate this 
problem, the leaders of the site teams have since been given the authority to 
remove such individuals, if detected, from the focus group samples and to replace 
them with alternates as needed. There have been no further problems with the 
inclusion of relatives in the focus group process, enabling the teams to gain the 
information the interviews are designed to acquire. 

Concluding the Data Collection Process: Building Consensus 

At the conclusion of a site visit, team members are often left feeling 
somewhat fragmented, as due to time constraints, no observer can see everything. 

To facilitate a shared sense of what is observed at the school, the 12HD (Teddlie & 
Stringfield, 1991)(see appendix 2), is used to bring about consensus regarding various 
school climate indicators. The instrument is scored by the group, and rates the 
school on a scale of one to seven (with 4.0 being a hypothetical state of "average") on 
various dimensions including: students' ability to make sense of their day, observed 
planned academic push, principal's presence throughout the school, coordination of 
school activities, and teachers' experimentation with new curriculum. 

An audiotape is made of the group while scoring the 12HD, as the scoring of 
this instrument tends to be very interactive. The group begins with a brief 
description of the school and its main features. Each item is scored individually by 
the team members', all members then discuss the group score, using examples 
gathered from the site visit to refute or support stated opinions. In addition, the 
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12HD offers a forum for events not observed by other team members to be 
acknowledged and recorded. 

Building a group consensus about individual school observations is an 
important culminating activity as it ensures that all observers leave the school site 
with a shared view of how they believe the school is performing. The strategy of 
consensus building with regard to school performance also begins the process of 
developing commendations and recommendations for the school, which is an 
integral part of the final report generated for each school visited. 

Going Beyond the Site Visits: Ancillary Benefits of SEAP-II 

As mentioned previously, the LDE began a total reorganisation coincident 
with the initiation of SEAP. A key concept in the reorganization of the LDE 
involved replacing the agency's rigid, top-down, bureaucratic, structure with a more 
horizontal organization to encourage across-function collaboration. The SEAP-II 
teams have become a prototype for this new way of work as it is the first large-scale 
effort to assemble teams of specialists from a range of units within the LDE and to 
assign them a common task: helping schools to identify their strengths and 
weaknesses. 

It is not unusual for a SEAP-II team to consist of staff drawn from such 
diverse administrative areas as the research unit. Title I, vocational education, 
special education, and early childhood sections. In addition, the teams cut across 
hierarchical levels — program managers, psychometricians, section heads, even 
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assistant superintendents. In some instances, assistant superintendents have 
worked under the direction of team leaders who were themselves program 
managers (entry-level professionals.) 

An important feature of the SEAP-II site visits is their ability to place LDE staff 
into schools. Although job qualifications for LDE staff are such that virtually every 
professional staff member at the Department is a former classroom teacher or 
administrator, many staff members are initially reluctant to participate in the site 
visits. However, once out in the field, the majority of participants find themselves 
invigorated by the opportunity to be back in schools. Even staff members whose 
jobs routinely call for them to be out in schools -- for example child welfare and 
attendance staff or Title I monitors — have for the first time been stepping outside 
their particular program bounds and find themselves acquiring different 
perspectives about schools. 

SEAP-II has also given the research staff involved with the LDE's education 
performance indicator program, the Progress Profiles, an opportunity to gather data 
useful in establishing concurrent validity regarding data collected by this unit. For 
example, for roughly 10 years the LDE has published data on student attendance, 
discipline, dropouts. The qualitative data generated by site-based observations, 
surveys, focus groups, and interviews has been used to validate or dispute what the 
indicators say about the school's disciplinary climate and student achievement. 

The SEAP-II site visit process also allows for the initiation of relationships 
between LDE staff and schools. This not only makes the new LDE goal of assistance 
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a reality, but also helps to lessen the fear a school commonly associates with a visit 
from the "State Department." Thus, SEAP-II has transcended its initial function as a 
process for gathering school level data to becoming (a.) an entity in which schools 
are examined on their own merits not just as statistics, and (b.) a means for bringing 
together LDE staff across functions and hierarchies with a common goal of school 
improvement. 




123 



Review of SEAP-II 25 



Works Cited 

Brookover, W.B., Schweitzer, J.H. , Schneider, J.M., Beady, C.H., Flood, P.K., 
& Wisenbaker, J.M. (1978). Elementary school social climate and school 
achievement. American Educational Research Journal, 15, 301-318. 

Brown, J.S., Collins, A., & Duguid, P. (1989). Situated cognition and the 
culture of learning. Educational Researcher, 18(1), 32-42. 

Oescher, J. & Brooks, C. R. (1993). Validation survey of the components of 
effective teaching. In, Louisiana Teacher Assessment Program for Interns: Spring 
1993 Field Test Evaluation Reports. Bureau of Professional Accountability, Office of 
Research and Development: Louisiana Department of Education. 

Patton, M.Q. (1990). Qualitative Evaluation and Research Methods. Sage : 
Newbury Park, CA. 

Schaffer, C.S., Nesselrodt, P.S., & Stringfield, S. (1994). The contribution of 
classroom observation to school effectiveness research. In Reynolds, D., Creemers, 
B. P. M. , Nesselrodt, P. S., Schaffer, E.C., Stringfield, S., & Teddlie, C., Advances in 
School Effectiveness Research and Practice. Pergamon Press: Great Britain. 

Stallings, J. (1980). Allocated learning time revisited, or beyond time on task. 
Educational Researcher, 9, 11-16. 

Teddlie, C. (1994). The integration of classroom and school process data in 
school effectiveness research. In Reynolds, D., Creemers, B. P. M. , Nesselrodt, P. S., 
Schaffer, E.C., Stringfield, S., & Teddlie, C., Advances in School Effectiveness 
Research and Practice. Pergamon Press: Great Britain. 



Review of SEAP-II 26 



Teddlie, C. & Stringfield, S. (1991). Observers as predictors of schools' 
effectiveness status. Elementary School Journal, 91 (4), 357-376. 

Teddlie, C., & Stringfield, S. (1993). School Do Make a Difference: Lessons 
Learned from a Ten Year Study of School Effects. Teachers College Press: New York. 




125 



Review of SEAP-II 27 



APPENDIX 1 



School: 



School Effectiveness and Assistance Pilot (SEAP) 

SCHOOL OBSERVATION CHECKLIST 

Date: 



Observer: 



Note: Not all items on this checklist will be appropriate to all schools. Every team 
member should complete sections 1-3. 

Sections 4-10 should be divided among the team members. Any notable 
observations relevant to unassigned sections should be recorded. 

Section I. Teacher 

1 . Note number/percent of teachers arriving: 

a. early ( 

b. at school starting time ( 

c. late ( 

Comments; 



2. Number of breaks allowed per day: 

3. Length of breaks allowed: 

4. Number of teachers leaving the lounge after break has ended. 

5. Number of teachers taking breaks in lounge: 

Comment: 
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6. Note type(s) of information posted on lounge bulletin board: 



7. Note comments, statements, and conversations of faculty, reflecting 
their attitudes and perceptions of their school in general, students, 
principal, local school district personnel, etc. 



8. Based on observations of teachers, do they appear to be satisfied 
with their jobs as: 

a. Professional educators: 



b. Teachers with this particular school: 



Further Comments: 



Section II. Principal Involvement 
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1. How often is the principal seen in the hallways during the day? 



2. In the classrooms? 



3. Comments: 



4. Describe the principal's rapport with the students (as indicated through 
observations). 



5. Describe the principal's rapport with the faculty (as indicated through 
observations). 



6. Are there visible signs of the principal's implementation of policies on 
personnel matters, student discipline, student achievement, in-service, 
safety and health, behavior codes, etc.? 



Section III. Assistant Principal(s) Involvement 
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1 . How often is the assistant principal(s) see in the hallways during the 
day? 



2. In the classrooms? 



3. Comments: 



4. Describe the assistant principal's rapport with the students (as 
indicated through observations). 



5. Describe the assistant principal's rapport with the faculty (as indicated 
through observations). 



6. Are there visible signs of the assistant principal's implementation of 

policies on personnel matters, student discipline, student achievement, 
in-service, safety and health, behavior codes, etc.? 
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Section IV. School Arrival 

1 . Note the proportion of students (a few, some, many, most, all) who 
arrive: 



early 


( 


%) 


at school starting time 


( 


%) 


late 


( 


%) 



2. Note the number of duty teachers when students arrive at school: 

3. Do there appear to be regimens or constraints placed on students' 
behavior? (e.g., strict structure placed on before-school behavior; 
students appear somewhat independent, with a few rules governing 
activities; students' activities are unrestricted.) 



4. Are there any security devices/regimens? (e.g., metal detectors, 
student ID badges, security officers) 



5. How are rules and regimens implemented? (e.g., teachers and/or staff 
use authoritative control; some guidance from staff, but students are 
self-disciplined) 



6. How do students respond to the rules and regiments? Describe their 
general before-school behavior. 



Further Comments: 
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Section V. Playground/School Grounds 



1 . Number of recess periods: Length: 

2. Monitoring of playground equipment/school grounds; number of duty 
teachers. 



3. Note amounts, types, and condition of playground equipment. 



4. Are there specific rules (formal or informal) regarding where students 
can congregate during recess? (e.g., students can/cannot go in their 
classroom(s) early; males gather near gym, girls near auditorium; 
playgrounds segregated by grade) 



5. Is there scheduled use of playground equipment, organized play, etc.? 
To what degree are the students independent in their playground 
activities? 



6. Are there specific playground rules and discipline policies in place? 
Describe each? If so, describe them. 
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7. 
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Do the students respond quickly to the school bell at the end of the 
recess period? 



Further Comments: 
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Section VI. Custodial Staff and Physical Appearance of School 

Somewhat Very 







Unclean 


Clean 


Clean 


1 . 


School grounds, playground(s) 


a 


b 


c 


2. 


Hallways, offices, bathroom(s) 


a 


b 


c 


3. 


Classrooms 


a 


b 


c 



4. How much interaction is there between faculty and/or students and 
custodial staff? 



5. What is the general attitude of faculty toward custodial staff? 

Negative Indifferent Positive 

6. What is the general attitude of students toward custodial staff? 

Negative Indifferent Positive 

7. Are some school buildings and facilities in need of repair, replacement, 
maintenance, etc.? 

walls 

building structure 

windows 

fences 

faculty desks, chairs 

student desks, chairs 

air conditioning system 

bathroom facilities 

water fountain(s) 



Further Comments: 
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Section VII. Cafeteria 

1 . What is the general demeanor of the cafeteria staff? 



2. How do students treat the cafeteria staff? 



3. Are the students allowed to talk during lunch? 



4. What other rules and regulations govern student behavior at lunch time 
(e.g., clean plate, disposal of trays, etc.) 



5. Describe the behavior of students at lunchtime. 



6. Are teachers required to eat lunch with their students? 



7. Are there cafeteria monitors? 
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[Applies to secondary schools] Are students allowed to leave campus 
at lunchtime? What rules govern student behavior after they finish 
lunch and before class resumes? (e.g., students must remain in 
cafeteria, students are restricted to a particular spot on campus, 
bathrooms are accessible/locked, etc.) 
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Section VIII. Auxiliary Classes 

1 . Are the P.E. classes organized with physical fitness in focus, organized 
play/exercise, or independent? 



2. Who teaches P.E. classes: 



3. What amount and type of P.E. equipment and resources are available? 



4. Who teaches music classes? 



5. What amount and type of music class equipment/resources are 
available? 



6. Who teaches art classes? 



7. What type of art is taught? What type of art materials are available? 
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8 . 
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Who teaches vocational classes? [Applies to secondary schools] 



9. What amount and type of vocational equipment and resources are 
available? [Applies to secondary schools] 



10. Does the guidance counselor come into the classroom regularly to lead 

discussion, provide information, etc. How often? 

What is discussed? 



1 1 . What type of special education services/resources are available? (e.g., 
autistic classes, signing for the hearing impaired, resource/remediation, 
etc.) 



1 2. Describe the attitudes of teachers and students of special education 
classes. Do they appear to be an integral part of the school culture? 



13. Are there scheduled visits and/or classes for students with the: 

a. social welfare worker 

b. school psychologist 

c. dietitian 

d. foreign language teachers : 

e. safety instructors 

f. health professional 

g. others 
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f. health professional 

g. others 



14. Do these visits appear to interfere with the regular class routine? 



1 5. What appears to be the attitude of the regular classroom teacher(s) 
toward these visits? 



16. In what other classes, extracurricular functions, activities, etc. are 
students involved? (e.g., plays, field trips, clubs/organizations, etc.) 



Further Comments: 
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Section IX. Hallways and Bulletin Boards 

1 . What is displayed on walls of hallways? (e.g., artwork of students, 
awards, posters, banners, announcements, etc.) 



2. What is displayed on bulletin boards in hallways? 



3. What are the subject matters of displays around the school? 



4. Do the displays have specific themes? 



5. How often do displays change? 



6. Who is responsible for displays on walls and bulletin boards? 



Further Comments: 
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Section X. Library 

1 . Is there a school library? 

2. Describe its physical attributes. 



3. Is there a librarian? 

If yes, is he/she full-time? 

halftime? 

less than halftime? 

4. If the librarian is less than full-time, what are his/her other duties? 



5. How are the students scheduled to visit the library? 



6. What is the general behavior of students in the library? 



7. Does the librarian have structured classes for students in library 
science? 

In general knowledge/various subjects 

If not, how is library time used? 



8. What resources are available in the library (furniture, books, AV 
equipment, periodicals, professional materials for faculty)? 
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9. Note the number and availability of 

a. library aids 

b. student helpers 

c. parental volunteers 

10. As indicated through observation, what is the general rapport of the 
librarian with students? 



With faculty? 




16 




Review of SEAP-II 43 



Section X. Other Observations 



School: Date: 

Observer: 
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APPENDIX 2 

12HD — Revised Version (OEI) 
Observers Exit Inventory (OEI) 
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School Name: 
Rater Name: _ 
Date: 



Directions 

The purpose of this taped meeting is to obtain an over-all, integrated impression of the school. 
Each team member should have already completed his/her own copy of the OEI before coming to 
the group meeting. Observers should not be concerned if their ratings for specific questions do 
not fit their overall impression of the school and are encouraged to discuss openly and informally 
any aspects of the school that seemed noteworthy to them. Each question can be discussed in any 
manner that suits the needs of the observers, but at some point should include a team agreed upon 
score from 1-7. Begin each tape with a description of the school, the size and make-up of 
the student body, and the community it serves. 

Scoring 

1 represents a judgement of "totally absent" or "not at all like this school." 

7 represents a judgement of "exemplary" or "present and working smoothly in virtually all 
observations." 



51. To what extent would you describe this school as having a serious academic atmosphere? 

1 2 3 4 5 6 7 

To what extent would you describe this school as being a friendly place for 
pupils/students and adults? 

1 2 3 4 5 6 7 

52. To what extent is this school one at which academic time is valued? 

1 2 3 4 5 6 7 

53. To what extent is this school one at which policy, curriculum, and school activities are 
consistently implemented and well coordinated? 

1 2 3 4 5 6 7 



S4. 



To what extent does the principal appear to be involved with the daily activities of 
teachers and students in the school? 



1 



2 3 



4 5 




Review of SEAP-II 45 



S5. To what extent are academic challenges, recognition, and rewards clearly posted? 

1 2 3 4 5 6 7 

T1 . To what extent does classroom teaching exhibit academic rigor and challenge? 

1 2 3 4 5 6 7 

T2. Rates of interactive teaching were (very low to very high). 

1 2 3 4 5 6 7 

T3. To what extend are classes experimenting with new curricula and instructional 
techniques? 



1 2 3 4 5 6 7 

ST1 . To what extent is this school one in which the total school experience of students 
appears to be meaningful? 

1 2 3 4 5 6 7 

ST2. To what extent is this school one in which the education of all students is valued equally. 

1 2 3 4 5 6 7 
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Association, April, 1998 San Diego, California 

The SEAP Process: Illustrative Case Studies from SEAP-II 

Overview 

The Louisiana School Effectiveness and Assistance Program (SEAP) is a two year pilot project 
(1996-97, 1997-98) that evolved from an earlier school performance model (Oescher, Black, Gunning, and 
Brooks, 1996). This earlier school performance model had been developed as part of the requirements for 
a Minimum Foundation Program (MFP) accountability system, within the Louisiana Department of 
Education (LDE), which had been authorized by the 1992 state legislature. The LDE’s response to this MFP 
accountability legislation included the development of academic performance measures that could by used 
in “equity” and “adequacy” applications. The SEAP program now includes statewide school accountability 
(SEAP-I), intensive school assessment (SEAP-II), and school improvement (SEAP-IH) activities. SEAP can 
best be conceptualized as a two track, parallel system of statewide school accountability and local school 
improvement activities. While the two tracks are linked through common research activities, each has 
independent goals. It is the purpose of the SEAP two-year pilot program to provide information to the 
Louisiana Board of Elementary and Secondary Education (BESE), the Accountability Commission created 
through House Bill 2068, and other educational policy makers regarding both school accountability and 
school improvement. 

The question that guided this research is: Can an intensive analysis of both process (qualitative) and 
product (quantitative) indicators of effectiveness be used to generate case study reports that assess the current 
effectiveness status of schools and make suggestions for improvement? 

Methodology 

SEAP-I 

The first activity undertaken by the researchers in the FY 1996-97 was the evaluation of an existing 
school performance model, which was generated using a mathematical model known as regression analysis 



(Oescher, Black, Gunning, & Brooks, 1996). This mathematical model, or more complicated multilevel 
models are often used in school effectiveness research (e.g. Brookover, et al., 1979; Mortimore, et al, 1988; 
Teddlie & Stringfield, 1993; Hill & Rowe, 1996) and in school indicator systems (e.g. Fitz-Gibbon, 1995, 
1996; Sanders & Horn, 1994; Webster & Mendro, 1997; Wilms, 1992). These regression models calculate 
school effectiveness indices (SEIs) after taking into consideration factors such as the socioeconomic status 
(SES) of the students attending the schools. The SEAP researchers produced (SEIs) for nearly 1,500 
Louisiana public schools. More than just a research exercise, the SEIs are used to target schools for intensive 
site-based assessment and ultimately, improvement. Each school’s performance was judged using two 
achievement measures (1) a standards-based Baseline Performance Indicator (BPI) and a Relative 
Performance Indicator (RPI), which takes into consideration six student and school intake characteristics. 
Each BPI is a school-level composite score reflecting student performance on all state-administered 
criterion-referenced tests (CRTs) for that site. The RPI, which also is a school -level composite score, 
reflects student performance on state-administered norm-referenced tests (NRTs) as well as CRTs. Other 
indicators (e.g. student participation index based on attendance, dropout, and/or other data) may be added 
at a later date. The SEIs calculated during the SEAP pilot are based on three years of achievement data 
(Spring 1995-1997) from the LDE’s Louisiana Educational Assessment Program (LEAP). These composite 
scores were produced by summing and averaging transformed subject area scores for the respective grade- 
level tests (Crone, Lang, Franklin, Halbrook, 1994; Crone, Teddlie, Franklin, 1995). The RPIs were 
calculated using two competing statistical models: school based regression and multilevel modeling (HLM). 

SEAP-n 

SEAP-II involves the intensive analysis of a select number of schools on both process (qualitative) 
and (quantitative) indicators of effectiveness. This information was then used to generate (case studies) 
school reports that assess the current effectiveness of the school and make suggestions for school 
improvement. The principal and superintendent of each school was presented the initial draft of the report 
in a meeting in which results of the analysis are discussed. The faculty of each school was then asked to 
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complete s tandar dized needs assessment forms, and the results from these needs assessments were used to 
refine the suggestions for school improvement. 

An abstract of the final report (case studies) was then given to the faculty of each school, and the 
faculty decided whether or not to participate in the school improvement process. If the school decided to 
participate in the process, then the LDE and Louisiana State University (LSU) affiliated external consultants 
jointly provided technical assistance as needed. Additionally, the LDE and LSU personnel assisted the 
schools in monitoring their progress under the school improvement plan. 

For Academic Year (AY) 1996-97, 12 schools (Cohort One Schools) were assessed from four 
volunteer districts. Some thirty LDE employees and external consultants affiliated with LSU were trained 
on the SEAP-II procedures in April, and the site visits took place in early May, 1997. Each five member 
team spent 1.5 days at each site, for a total of 7.5 person days per site. The activities included observing in 
the classroom (24 hours per school), interviewing the principal, conducting focus groups with students and 
teachers, completing school checklists through observation in the school and on the grounds, and distributing 
and collecting attitudinal questionnaires from parents, teachers, students and the principal. 

After the site visits, team members met in Baton Rouge at the LDE building in June to discuss their 
experiences and to write school analysis reports. These case studies were written in accordance with the 
procedures advocated by Yin (1994) and the Teddlie and Stringfield (1993) case studies. The final reports 
were completed by September and visits with each of the faculties occurred at this time. The final reports 
cite strengths of each school and make specific recommendations for improving identified areas of 
weaknesses. Each school then developed its own school improvement plan using the final LDE/LSU report 
as a starting point in conjunction with a LSU analysis of the school needs assessment. 

School Report (Case Study) Format 

The twelve (12) case studies were guided by the following outline of information. (See Table 1) The 
main section headings included School Context, School Effectiveness Indicators, District Level Factors, 
School Administration and Other Factors at the Level of the School, Classroom Teachers and Other Factors 



at the Level of the Class, Life in the School at the Student Level, Results from the School’s Needs 
Assessment, Commendations for the School, and Recommendations for School Improvement. 
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Table 1 

SEAP-n REPORT 
TABLE OF CONTENTS 

I. SCHOOL CONTEXT 

A. DESCRIPTION OF THE SCHOOL PHYSICAL ENVIRONMENT 

B. DESCRIPTION OF THE AREA WHERE THE SCHOOL IS LOCATED 

C. DEMOGRAPHIC INFORMATION 

n. SCHOOL EFFECTIVENESS INDICATORS 

A. General School Demographic and Participation Indicators9 

B. Student Performance on Norm-Referenced Tests (NRTs)lO 

C. Student Performance on Criterion-Referenced Tests (CRTs)12 

D. Graduates, ACT Test Results and First Time Freshman Profiles 15 



m. DISTRICT LEVEL FACTORS 



A. Superintendent and Other General District Factors 

B. Relevant District Level Policies 

IV. SCHOOL ADMINISTRATION AND OTHER FACTORS AT THE LEVEL OF THE SCHOOL 

A. Tenure of Principal and Other administrators 

B. Relevant School Level policies 

C. Principal attitudes 

D. Teacher perceptions of Principal and Administration 

E. Student and Parent Perceptions of the Principal and Administration 

V. CLASSROOM TEACHERS AND OTHER FACTORS AT THE LEVEL OF THE CLASS 

A. TEACHER RETENTION, TEACHER ABSENTEEISM 

B. TEACHER ATTITUDES 

C. Teacher Performance in the Classroom 

VI. LIFE IN THE SCHOOL AT THE STUDENT LEVEL 

A. Student Attitudes 

B. Parent Perception of Student life at the School 

C. School Observations 



VII. RESULTS FROM SCHOOL’S NEEDS ASSESSMENT 

A. Faculty perceptions of the School’s Strengths 

B. Faculty Perceptions of the School’s Weaknesses 

C. Faculty’s Suggestions for Improving the school 



VIII. CO 






NDATIONS FOR THE SCHOOL 



EL RECOMMENDATIONS FOR SCHOOL IMPROVEMENT 




Section and specific information to be included was detailed under the section heading. Prompts 
were provided listing questions to be answered under each heading. Team members worked together to write 
some sections and worked independently on other items. A team consisting of report writing specialists 
reviewed and edited the rough draft of the reports for continuity and consistency across all 12 reports. 

Case Study Demographics 

One application of case studies is to describe the real-life context in which an intervention or 
innovation has occurred. Yin (1989) defines case studies as examining a range of complex social phenomena 
and representing a holistic approach to research. In addition ,Yin felt that case studies are process oriented 
describing the mechanism of the intervention or innovation. Quantitative research gives parameters and 
measurements to set criteria while qualitative research (case studies) allows for a broader discussion of 
perceptions, attitudes, and interpretations of situational conditions by members of the organization under 
study. By utilizing both quantitative and qualitative methods in this research, confirmation of quantitative 
and qualitative findings are possible. The quantitative section of this study is enhanced and expanded by 
case studies which give greater detail to the relationships found within the schools. The following paper 
includes quantitative summaries across all 12 case studies regarding demographics, teacher performance in 
classroom observations, school commendations and recommendations. In addition to these quantitative 
summaries, qualitative information contained in two case studies is presented in narrative form. 

The four districts in this study are from four distinctly different parts of the state. There is a disparity 
of urbanicity, school population, teacher/student ratio, and geographic location across the 12 schools. Each 
district in the study volunteered their schools to be part of the statewide accountability field test. This 
resulted in two districts volunteering only rural schools and no high schools being represented in the SEAP-II 
Cohort I schools. In order to get the maximum amount of candid information, personnel and students at the 
school level were promised anonymity. The names of the districts, schools, and personnel have been 
assigned pseudonyms. Table 2 presents the participants in the study by their pseudonyms. Listed for each 



school are the districts, schools, location, student/teacher ratio, grade configurations, ethnicity, and 
socioeconomic status (based on percentage of free lunch) for each school. The teacher section includes not 
only classroom teachers, but support personnel at the schools. These 12 schools are representative of the 
types of Pre- Kindergarten through Grade Eight schools found in Louisiana. They include a variety of sizes 
(range from 159 to 749) and school configurations, along with a varied range of teacher/pupil ratios that are 
reflective of the auxiliary and ancillary personnel assigned to the school for support and assistance. The 
ethnicity level of these sample schools range from a 90% to a 6.7% white population. The state public 
school population is 51% white. African- American children comprise the other 48%, with Hispanics, 
Asians and other ethnicities completing the whole. The SES status based on free lunch eligibility ranges 
from 25% to 82.3%, with the state average being 57%. 
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Table 2 

School Effectiveness and Assistance Program 
SEAP-II Field Test Participants 
Demographic Data 



K)istrict 


School 


Location 


Population 
Student Teacher 


Level 


Ethnicity 

White 


SES 

Free Lunch 


■Mchafalaya (A) 


#1 


Rural 


231 


18 


5-8, NG 


7.8% 


77% 


#2Creole 


Rural 


309 


28 


PK-4, NG 


6.7% 


82.3% 


#3 


Rural 


399 


31 


5-8, NG 


50% 


69% 


MBaratana (B) 


#1 


Rural 


749 


40 


4-6 


84.5% 


11.3% 


#2 


Urban 


562 


39 


K-5 


68.3% 


25% 


#3 


Urban 


581 


45 


K-5 


59% 


45% 


1 Comite (C) 


#1 


Rural 


184 


15 


K-8 


88.3% 


34.5% 


#2 


Rural 


551 


40 


K-4 


57.6% 


66% 


#3 


Rural 


159 


15 


5-8 


38% 


71% 


■Delta (D) 


#1 Bayou 


Rural 


466 


34 


4-6 


73% 


59% 


#2 


Suburban 


436 


29 


3-5 


90% 


38.8% 


#3 


Suburban 


381 


38 


K-7 


87.5% 


32.4% 
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Quantitative Results 



School Climate Scale 

The instruments for school climate assessment were modified versions of Brookover and colleagues 
(1979) and other school climate questionnaires. Surveys were administered to a sample of parents and 
students randomly and representatively selected from the school population. The entire faculty and teaching 
staff completed the surveys as well as each principal at the 12 school sites. The survey items were worded 
in a parallel fashion, allowing for comparisons across the constituencies and the following variables: General 
Reputation of the School, Safe and Orderly Environment, Expectations, Academic Norms, Academic 
Efficacy; Organ izat ion collaboration and Organization Leadership; Quality of Instruction, Parent/School 
Relationship, Staff Development, and Job Satisfaction. 
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Table 3 

Common Results Across the 12 Case Study Schools 
Mean Attitudinal Responses of Students, Parents and School Staff, 
by Type of Respondent 



Respondent 


Student 


Teacher 


Principal 


Parent 


General Reputation of the School 


na 


3.72 


3.92 


3.64 


Climate I: Safe and Orderly 
Environment 


3.37 


3.90 


4.57 


3.57 


Climate II: Expectations 


4.10 


3.63 


3.60 


3.76 


Climate III: Academic Norms 


4.00 


3.80 


3.85 


2.90 


Climate IV: Academic Efficacy 


3.93 


4.02 


4.43 


3.93 


Organization I: Collaboration 


na 


3.72 


433 


na 


Organization II: Leadership 


na 


3.87 


4.40 


3.63 


Quality of Instruction 


4.08 


4.15 


4.26 


3.84 


Parent/School Relationship 


3.77 


3.28 


3.98 


3.88 


Staff Development 


na 


3.61 


4.08 


na 


Job Satisfaction 


na 


3.75 


433 


na 



Source: Site-level Surveys of Parents, Students, and School Staff, Spring 1997 
na=Not Applicable. 1 Based on one survey item. 
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Table 3 presents a summary of the mean responses by the constituency and the variables. All scores 
on this table are based on five point scales, on which “5" indicates the most positive response and “1" 
indicates the least favorable response. An average score is approximately 3.50. All scores, except for 
“General Reputation of School,” are based on multiple items; for example the score for “Safe and Orderly 
Environment” is the average score for five items that measured different aspects of that component. 

Across all 12 schools, a discrepancy between the responses of teachers and principals and that of 
students and parents became evident. Teachers and principals consistently expressed much lower 
expectations for students than did the students themselves and their parents. In contrast, teachers and 
principals reported that the quality of education at their schools was better than the opinions expressed by 
students and parents. This discrepancy between constituency responses was recognized when 
recommendations for each school were written; most schools were encouraged to improve upon their 
school/parent/community relationships. 

Louisiana Components of Effective Teaching 

Louisiana Components of Effective Teaching (LCET) is a modified version of an instrument 
developed for the Louisiana Teacher Assessment Program. The main difference between the two, is that the 
assessment version rates teachers’ behaviors on a two point scale, pass/fail; while the SEAP version of the 
instrument uses a four point scale to obtain a great degree of variance in scoring. 

The LCET focuses on two domains, management and instruction. Within the management domain 
are components which examine the learning environment, maximization of time, and management of learner 
behavior. Within the instruction domain are components which examine the teacher s ability to deliver 
instruction effectively, the teacher’s presentation of appropriate content, the teacher s provision of 
opportunities for pupil involvement in the learning process, and how the teacher assesses pupil progress. 
Each component is further subdivided into attributes, which are what is rated by the observer. 

Table 4 presents the aggregated results for the Louisiana Components of Effective Teaching (LCET) 
across all 12 schools. Each school received this data summed for the entire school, and then broken down 
to each grade level. The LCET is scored using a four point scale. A “4" indicates that the teacher does an 
outstanding job and no area for improvement is readily identifiable. A “3" indicates the teacher consistently 
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meets and sometimes exceeds expectations , however, performance can be improved, but current practices 
are clearly acceptable. A “2" indicates the teacher’s performance sometimes, but not always meets 
expectations and improvement activities are required for performance to consistently meet standards. A “1" 
indicates the teacher’s performance is not acceptable and improvement activities must be undertaken 
immediately. A score below a 2.70 for either a component or attribute represents an area of weakness that 
the school needs to address 
with targeted staff development. 
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Table 4 

Common Results Across the 12 Case Study Schools 
Teacher Ratings on Components of Effective Teaching - Mean Scores 



Domains, Components, Attributes 


12 Schools 


Domain II. Management, Component A 

The teacher maintains an environment conducive to learning. 


2.94 


II(A1). Organizes available space, materials, and/or equipment to facilitate learning. 


2.97 


II(A2). Promotes a positive learning climate. 


2.92 


Domain II. Management, Component B 

The teacher maximizes the amount of time available for instruction. 


2.66 


II(B 1). Manages routines and transitions in a timely manner. 


2.82 


II(B2). Manages and/or adjusts time for activities. 


2.53 


Domain IL Management, Component C: The teacher manages learner behavior to provide 
productive learning opportunities. 


2.78 


II(C1). Establishes expectations for learner behavior. 


2.79 


II(C2). Uses monitoring techniques to facilitate learning. 


2.77 


Domain III. Instruction, Component A: The teacher delivers instruction effectively. 


2.55 


III(Al). Uses technique(s) which develop(s) lesson objective(s) 


2.59 


III(A2). Sequences lesson to promote learning. 


5.60 


III(A3). Uses available teaching material(s) to achieve lesson objective(s). 


2.55 


III(A4). Adjusts lesson when appropriate. 


2.23 


Domain III. Instruction, Component B: The teacher presents appropriate content. 


2.96 


III(Bl). Presents content at a developmental^ appropriate level. 


3.03 


III(B2). Presents accurate subject matter. 


3.22 


III(B3). Relates relevant examples, unexpected situations, or current events to the content. 


2.63 


Domain III. Instruction, Component C: The teacher provides opportunities for student 
involvement in the learning process. 


2.52 


III(Cl). Accommodates individual differences. 


2.06 


III(C2). Demonstrates ability to communicate effectively with students. 


2.78 


III(C3). Stimulates and encourages higher order thinking at the appropriate developmental 
levels. 


2.33 


III(C4.) Encourages student participation. 


2.76 


Domain ID. Instruction, Component D: The teacher assesses student progress. 


2.64 


III(Dl). Monitors ongoing performance of students. 


2.71 


III(D2). Provides timely feedback to students regarding their progress. 


2.56 
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Based upon the overall results, the 12 schools performed poorly in the following 4 components: 
Management Component B: The teacher maximizes the amount of time available for instruction (2.66); 
Management Component C: The teacher manages learner behavior to provide productive learning 

opportunities (2.55); Instruction, Component C: The teacher provides opportunities for student involvement 
in the learning process (2.52); and Instruction, Component D: The teacher assesses student progress (2.64). 
In addition, under the Instruction Component: : The teacher provides opportunities for student involvement 
in the learning process, two attributes were consistently rated low: Accommodates individual differences 
(2.06) and stimulates and encourages higher order thinking at the appropriate developmental levels (2.33). 
The attribute “adjusts lesson when appropriate” under the Instruction Component A was another area where 
teacher performance was low (2.23). The scores on these three attributes are in line with other programs in 
the state which use this instrument, most notably the Louisiana Teacher Assessment Program (LTAP). 

Recommendations and Commendations 

The following two sections consist of an aggregated analysis of all twelve schools’ commendations 
and recommendations. The methodology used to analyze each schools’ recommendations and 
commendations involved using Lincoln and Guba’s (1985 ) method of unitizing and categorizing the data. 
Each item was divided into units which were then classified by category code, and then a frequency count 
for each category was tabulated. 

Commendations 

Tables 5a and 5b show the results of the content analysis of the commendations across all 12 schools. 
Table 5a reports the commendations at the school level and Table 5b shows the frequency of commendations 
across school. 

The most frequently recognized commendation was the strength of the school faculty. These areas 
included the commitment of the faculty to furthering their knowledge and professional credentials, 
participation in the school decision-making process, and teacher stability in the schools. Faculty set high 
expectations for learner behavior creating a positive school climate. 
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Table 5a 

SEAP-II Cohort-I Schools 
Types of Commendations Across Schools 
Parti 



NRT/CRT 


Faculty 


Community 

Patterns 


Attitudinal 

Scores 


Instruction 


School 

Operations 


A, 1st, # 2 


A, 3rd, # 4 


A, 2nd, # 2 


A, 1st, # 1 


B, 2nd, # 7 


B, 1st, # 8 


A, 1st, # 3 


A, 3rd, # 5 


B, 1st, # 1 


A, 2nd, #3 


B, 2nd, # 9 


B, 2nd, # 6 


A, 2nd, # 1 


B, 1st, #4 


B, 2nd, # 1 


A, 3rd, # 6 


C, 1st, # 6 


B, 3rd, # 1 


A, 2nd, #4 


B, 1st, # 5 


B, 2nd, #3 


B, 3rd, # 4 


L, 1st, # 5 


C, 2nd, # 2 


A, 3rd, # 3 


B, 1st, # 6 


B, 3rd, # 2 


C, 1st, # 1 


L, 1st, # 7 




B, 3rd, # 3 


B, 1st, # 7 


L, 1st, # 4 


L, 3rd, # 3 






C, 1st, #2 


B, 2nd, # 8 


L, 1st, # 6 








C, 1st, #3 


C, 1st, # 8 










C, 3rd, # 2 


C, 2nd, # 1 










L, 1st, # 1 


C, 2nd, # 3 










L, 1st, # 2 


C, 3rd, # 1 










L, 1st, #3 












L, 3rd, # 1 













Part II 



Physical 

Facilities 


Administrate 

n 


Discipline 


Achievement 


Attendance 


State 

Standards 


B, 1st, # 2 


C, 3rd, # 3 


A, 1st, # 4 


B, 1st, # 3 


A, 3rd, # 2 


C, 1st, # 4 


B, 2nd, # 4 


C, 3rd, # 4 


A, 3rd, # 1 


B, 2nd, # 2 






B, 2nd, # 5 


L, 1st, # 8 


C, 1st, #5 








C, 1st, U 1 


L, 3rd, # 2 
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Testing results indicated that several of the schools showed an increase in CRT and NRT scores. 
Pupils in many cases tested above the national median on state administered standardized tests. Attitudinal 
scores reflected the students positive attitudes toward their school experiences. Many of these scores were 
reflected by the students responding to caring attitudes of their teachers. Expectations were high concerning 
school and the opportunities it offered, which is a positive indication of improved student performance. 
Schools were commended for encouraging the community to play an active part in the school. This was 
evidenced by the successful partnerships with businesses in the community. 

Effective discipline and improved attendance was reflected by a decrease in the student suspension 
rate. Most discipline problems were handled in the classroom. Daily operations at several of the schools 
were well-organized and maximized the use of instructional time. Data from the time-on-task scans 
indicated high scores and effective instruction in many of the 12 schools. 



Table 5b 

SEAP-II Cohort-I Schools 
Frequency of Recommendations Across Schools 



Category 


Frequency 


Norm/Criterion Referenced Tests 


13 


Faculty 


11 


Community Patterns 


7 


Attitudinal Scores 


6 


Instruction 


5 


Physical Facilities 


4 


School Operations 


4 


Discipline 


3 


Administration 


3 


Achievement 


2 


Attendance 


1 


State Standards 


1 
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Recommendations 



Table 6a and 6b show the results of the content analysis of the recommendations across all 12 
schools. Table 6a focuses on recommendations targeted at the school level and Table 6b focuses on 
recommendations targeted at the district level. Breaking this analysis down even further is Table 7 which 
reports the frequency of recommendations across schools. The most frequently made recommendation had 
to do with leadership, with every school receiving at least one recommendation under this heading. The 
majority of recommendations in this area looked at scheduling/planning/protecting academic time; school 
and classroom discipline policy; and increasing principal observations along with teacher feedback. 

Every school was also cited for needing staff development as a recommendation for school 
improvement. Most of the suggestions centered around improving instructional methods, classroom 
management, and addressing the individual needs of all students and special needs students. Three-quarters 
of the schools’ recommendations centered around curriculum as an area needing improvement. These 
recommendations focused on language arts, reading and math instruction becoming a more hands on, 
interactive experience for students. 

Half of the recommendations made for the schools regarded improving parental involvement and 
communication. This was primarily in response to the variance between the responses of the parents and 
those of the teachers and principals on the attitude surveys. One third of the schools had recommendations 
that targeted the low expectations that faculty and staff appeared to have for students. This was evidenced 
in the results of the classroom observations and the attitudinal survey responses. 

It is important to note that professional development was cited as a recommendation in all 12 
schools. Targeted staff development, built upon the goals of school improvement plans is a concrete way 
in which schools can begin to address their shortcomings and move toward the future. 
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Table 6a 

SEAP-n Cohort-I Schools 
Types of Recommendations Across Schools 



® Leadership 


Professional 

Development 


Curriculum 


Parent Involvement 
Communication 


Expectations 


Physical Site 


Other 


1st, #1 


A, 1st, #7 


A, 1st, #5 


A, 3rd, #3 


A, 1st, #3 


A, 1st, #8 


B, 3rd, #8 


[A, 1st, #2 


A, 2nd, #3 


A, 1st, #6 


B, 1st, #5 


A, 1st, #4 


D, 1st, #6 




2nd, #1 


A, 2nd, #5 


A, 2nd, #4 


B, 2nd, #5 


A, 2nd, #2 






^A, 2nd, #3 


A, 3rd, #4 


A, 3rd, #11 


C, 3rd, #7 


C, 2nd, #5 






3rd, #1 & 2 


A, 3rd, #6 


B, 1st, #7 


D, 1st, #5 


C, 3rd, #2 






^S., 3rd, #3 


A, 3rd, #7 


C, 2nd, #2 


D, 3rd, #4 








pA, 3rd, #5 


A, 3rd, #8 


C, 2nd, #8 


D, 3rd, #5 








^A, 3rd, #10 


A, 3rd, #9 


C, 2nd, #10 










pB, 1st, #2 


B, 1st, #1 


C, 3rd, #4 










Lb, 1st, #3 


B, 1st, #4 


D, 1st, #7-1 










fB, 1st, #6 


B, 2nd, #1 


D, 2nd, #5 










Lb, 2nd, #2-1 & 


B, 3rd, #3 


D, 2nd, #6 










f B, 2nd, #2-3 


B, 3rd, #6 


D, 3rd, #6 










■p, 2nd, #3-1 & 


C, 1st, #3-1 


D, 3rd, #7 










J_B, 2nd, #4 


C, 1st, #3-2 












|B,3rd,#2 


C, 2nd, #4 












B, 3rd, #3 


C, 2nd, #7 












PB,3rd,#4 


C, 2nd, #9 












B, 3rd, #5 


C, 3rd, #3 












^B,3rd,#7 


C, 3rd, #5 












1 C, 1st, #4 


D, 1st, #2 












■ C, 1st, #5 


D, 1st, #3 












1c, 2nd,#l 


D, 1st, #4 












^lc, 2nd, #3 


D, 1st, #7-2 












L C, 2nd, #6 


D, 1st, #8 












I C, 3rd, #1 


D, 2nd, #1 












Ad, 1st,# 6 


D, 2nd, #2 












^ D, 2nd, #7 


D, 2nd, #9 












A p, 3rd, #2-1 & 


D, 3rd, #3 













1 0 

ERIC 



19 IRQ 



BEST COPY AVAILABLE 



I 

I 



Table 6b 

SEAP-II Cohort-I Schools 
Types of Recommendations Across Schools 



leadership 


Staffing 


Management 


Physical Site 


Staff 

Development 


Programs / 
Pupil Teacher 
Ratio 


Technology 

Resources 


|D, 1st, #1 


A, 1st, #9 


D, 2nd, #4 


C, 1st, #2 


A, 3rd, #4 


C, 1st, #1 


B, 3rd, #1 


^D, 2nd, #8 


A, 1st, #10 




C, 3rd, #8 


D, 3rd, #3 


C, 3rd, #6 






A, 3rd, #12 




D, 3rd, #1 










D, 2nd, #3 














D, 3rd, #4 














D, 3rd, #8 
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Table 7 

SEAP-II Cohort- 1 Schools 
Frequency of Recommendations Across Schools 



Category 


Frequency 


Leadership 


29 


Scheduling/Planning/Protect Academic Time 


7 


School and Classroom Discipline Policy 


7 


Classroom Observations/Feedback 


4 


School Improvement Team 


2 


Improve Communication/Collaboration 


2 


Class Size 


2 


New Principal 


1 


Reward Pupil Success 


1 


Homework Plan 


1 


Special Education 


1 


Student Involvement 


1 


Professional Develooment 


29 


Instructional Methods 


7 


Individual Needs/Special Needs Students 


7 


Classroom Management 


4 


Structure Opportunities/Staff Development Coordination 


4 


Higher Order Thinking Skills 


2 


Technology, Discipline, & Data Analysis 


2 


Learning Styles 


1 


Teachers on Self-Esteem 


1 


Cooperative Teaching 


1 
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Table 7 



Frequency of Recommendations Across Schools (Cont’d) 



Category 


Frequency 


Curriculum 


14 


Language Arts 


3 


Reading 


2 


Math 


2 


Curriculum 


2 


Gifted and Talented 


1 


Vocational Education 


1 


Extra Curricular Activities 


1 


Computer 


1 


Social Studies, Study Skills 


1 


Parental Involvement/Communication 


7 


Expectation? 


6 


Physical Site 


3 


Q.th£JC 


I 


TOTAL 


89 
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Case Studies 



The following excepts from two illustrative case studies (school reports) of Bayou Elementary and 
Creole Elementary are indicative of the information contained within the case studies about school context 
and the leadership/administration of the schools (Section I. SCHOOL CONTEXT and Section IV . SCHOOL 
ADMINISTRATION AND OTHER FACTORS AT THE LEVEL OF THE SCHOOL). These two schools 
are similar in context, but dissimilar in leadership/administration of the school. Table 8 depicts the 
comparison of the two schools in these two areas: 

Table 8 

Comparison of Bayou Elementary and Creole Elementary Schools on 
School Context and Leadership and Administration Components 



SCHOOL CONTEXT 


LEADERSHIP|ADMINISTRATION 


Category 


Bayou 

Elementary 


Creole 

Elementary 


Category 


Bayou Elementary 


Creole Elementary 


Location 


Rural 


Rural 


Ethnicity 


White 


White 


Level 


K-7 


PK-4.NG 


Gender 


Male 


Female 


Population 

Students 


385 


309 


Teaching 

Experience 


10 Years 


18 Years 


Ethnicity 
J White 


73% 


6.7% 


Administrative 

Experience 


23 Years 


2 Years 


SES/Free Lunch 

i 


59% 


82.3% 


— B 


M— 




1 

Population 

i Te “ hers . 


35 


28 








l 

% of Faculty with 
| Master’s Degree or 
J Higher 


37% 


30% 


IBM 
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School Context Indicators 



Table 8 provides school summary descriptive information from the Progress Profiles 2 (School 
Report Cards). Both Bayou Elementary and Creole Elementary are located in two of the southernmost 
coastal, rural Parishes in the State. Bayou Elementary, a one-time K-12 school has been truncated to a K-7 
school serving a population of approximately 385 students from mostly working-class backgrounds. Creole 
Elementary serves pupils in grades PK through 4. Enrollments at Bayou Elementary have been relatively 
stable in recent years, averaging between 370 and 380 students over the past three years, with 1996-97 at 
385. As the table notes, enrollments at Creole Elementary declined about 29% between 1993-94 to 1995-96; 
that is, enrollment decreased from 483 pupils in 1993-94 to 342 pupils in 1995-96, and became stable during 
1996-97 at 309. The population at Bayou Elementary is predominately white (73%), whereas the student 
population at Creole Elementary is predominately non-white (92.3%). The free lunch figures indicate that 
Bayou Elementary (59%) has a smaller number of at-risk students as compared to Creole Elementary 
(82.3%). Both of these at-risk percentages are higher than the state’s at-risk population (57%). 

The teacher workforce at Bayou Elementary (35) is larger than Creole (28), but this is reflective of 
the 76 additional students which includes the deaf education students that require special needs teachers. 
Approximately 37% of the faculty at Bayou held at least a master’s degree in 1995-96, down somewhat from 
the year before (47.06%). Approximately 30% of the faculty at Creole held at least a master’s degree in 
1995-96. The percent of faculty (including teachers, guidance counselors, and administrators) who have 
earned a master’s degree or higher is a reflection of the extent to which the instructional staff have made a 
commitment to furthering their own knowledge and professional credentials. 

Bayou Elementary and Creole Elementary are both located in rural communities and have at-risk 
student populations. The population at Creole Elementary has a higher percentage of at-risk students than 
Bayou Elementary. The student population has seen a decline over the past several years at Creole, whereas 



1995-96 Louisiana Progress Profiles District Composite Report , Delta Parish. A technical report published March 1997, by the 
Louisiana Department of Education, Office of Research and Development, Bureau of School Accountability. 
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Bayou Elementary has stayed relatively stable. The instructional staff at both schools is comparable in their 
commitment to furthering their professional development. These two schools have very similar school 
context variables , but as this study will show other factors will influence the strengths and weaknesses in 
these schools. 

Leadership/Administration Indicators 

In Table 8, the leadership /administration indicators compare the demographic data on the principals 
from Bayou Elementary and Creole Elementary. Bayou Elementary’s principal is a white male with ten 
years of teaching experience and 23 years of administrative experience. In contrast. Creole Elementary’s 
principal is a white female with 1 8 years of teaching experience and two years of administrative service. 
The following case study will further contrast the differences and similarities between the administration 
in Bayou Elementary and Creole Elementary. 

Case Study Reports 

The entire case study reports were presented to the District Superintendents and the principals at a 
meeting to decide the focus of the school improvement plan for the schools. The case study approach 
provides the school system with a contextual background and an actual picture of what is occurring within 
the schools. The feedback to the schools was in a narrative format and provides examples of justifications 
for the improvement recommendations. The following excepts from two of the twelve case studies are 
based on the triangulation of both qualitative and quantitative data gathered from the SEAP site visits to the 
schools. References are made throughout the case studies to mean scores from attitudinal responses to 
school climate surveys administered by the SEAP site visit team to the principal, teachers, pupils, and 
parents. All items are based on five-point scales, with five indicating the most positive response and one 
indicating the least positive. 

Bayou Elementary School 

Bayou Elementary School is located in one of the southernmost coastal, rural Parishes in the State. 
Bayou Elementary is one of the oldest schools in operation in Delta Parish. The campus, which is located 
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on LA 1, facing Bayou Lafourche, is a mix of the old and new, permanent and temporary buildings on a plot 
of land carved from an old sugar cane field. The focal point of the campus is a two-story, clapboard building 
erected in 1918, with wide halls and high ceilings. Though an annex with a combination of 
cafeteria/auditorium was added in the 1980's, the two permanent buildings have comparatively few 
classrooms between them. Therefore, the majority of students are housed in an assortment of temporary 
buildings, clustered at the rear of the main building. 

Delta, one of the southernmost parishes in Louisiana is located west of the Mississippi River and 
south of New Orleans in the heart of the Mississippi delta. Bayou Lafourche, the parish’s principal 
waterway, traverses the parish from the north to the south and serves as an economic corridor for the region. 
From Thibodaux in the northernmost tip of the parish to Golden Meadow and Fourchon in the south, most 
of the parish’s residents live, work, and attend school along LA 1, the state highway that skirts the Bayou’s 
western bank. 

Though Delta has been described by all interviewed as a neighborhood school, there is limited 
community involvement in the school, partly because the school draws from a wide geographic area, with 
few businesses in the immediate area. Parents seem to value education because they reportedly respond 
quickly and supportively to contacts from teachers and attend open houses and other school functions. 

Delta Parish is somewhat above the state average financially, but slightly below the state average 
economically. For example, about three in ten households (30%) had average annual incomes of less than 
$15,000 in 1990, as compared to a state average of 36.3%. The district is in fact solidly middle class, with 
approximately 60% of Delta households earning incomes between $15,000 and $50,000 as compared to a 
state average of 48.3%. On the other hand, 43.8% of adults held less than a high school education as 
compared to 27.4% for the state as a whole. The slight imbalance in economic and educational performance 
indicators is not surprising in this region because residents have traditionally earned their livelihood in the 
fishing and petroleum industries where semi-skilled workers were long able to earn good wages. Not 
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surprisingly, 40.8% of the parish’s workforce in 1990 were blue collar workers (U.S. Bureau of the Census, 
1990). 

Risk factors for children and youth are somewhat lower in Delta than for the state as a whole. Some 
15% of households were headed by single parents in 1990 as compared to 19.1% statewide. Thirty percent 
of Delta residents were living below the poverty level that year (the state average was 23.6%), and the 
parish’s teen pregnancy rate was 15.8% as compared to 17.6% for the state as a whole (U.S. Bureau of the 
Census). 

Delta Parish is one of the state’s larger public school districts, with roughly 16,300 students in grades 
K-12 enrolled in the district’s 28 public schools in 1995-96. Public school students here outnumber 
nonpublic students more than 6 to 1. The parish’s seven nonpublic schools (all Catholic) enrolled 
approximately 2,600 students in grades K-12 in 1995-96 (Bulletin 1462, Louisiana School Directory). The 
Delta Parish system employed nearly 1,200 faculty in 1995-96 (Bulletin 1462, Louisiana School Directory) 
at an average annual salary of $26,838 compared to an average $24,873 statewide (Bulletin 1472, Annual 
Financial and Statistical Report, 1995-96). Teacher salaries in Delta rose to $26,838 in 1995-96 from a low 
of $22,736 in 1993-94 (Bulletin 1472, Annual Financial and Statistical Report, 1995-96 and 1993-94). 

In 1995-96, the district spent $4,359 per student in average daily membership compared to a state 
average of $4,468 (Bulletin 1472, Annual Financial and Statistical Report, 1993-94). However, district 
funding per student is expected to jump nearly seven-fold (from $5 to $34 per student) when a recently 
approved sales tax increase goes into effect. 

Bayou Elementary, a one-time K-12 school has been truncated to a K-7 school serving a population 
of approximately 385 students from mostly working-class backgrounds. The student body appears somewhat 
better off economically than the parish average, judging from the free/reduced lunch rate of 32.4% for Bayou 
Elementary. According to the principal, about half the students’ parents have completed high school, and 
the great majority of families are above the poverty level. 

Approximately 15% of the children attending Bayou Elementary in 1996 were special education 
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students, with fewer than 1% identified as “gifted and talented.” It appears that many of the special 
education children at Bayou are hearing-impaired students enrolled in a deaf education program that the 
district launched at the school some three years ago. The hearing-impaired students spend most of their day 
in a regular education classroom, and are pulled out for resource. They also have an “adaptive physical 
education teacher” who is the only physical education teacher on site. Only .5% of students at Bayou were 
identified as Limited English Proficient in 1996. 

Tenure of the Principal and Other Administrators 

Mr. John Boudreaux, a native of Delta Parish, is a family man whose wife has also spent a career 
working for education at the university level. Mr. Boudreaux has served his entire educational career within 
the district, rising through the teaching ranks from his first post as a fifth grade teacher to become principal 
of Bayou Elementary. He has been principal at Bayou for 23 years. There are no assistant principals, 
administrative assistants, or guidance counselors at Bayou. 

Mr. Boudreaux has completed the DROP program and is going into his second year after DROP. 
At this point in his profession, he is relatively disengaged from the instructional component of the school 
and, with the exception of serious problems, takes a “hands off’ approach to his teachers. In regards to 
school safety/discipline, teacher focus group respondents said that they handle most discipline problems 
themselves, at the classroom level. 

The SEAP team also perceived the principal to be “in a holding pattern” insofar as instructional 
leadership is concerned. He was not visible within the classrooms during the site visit and, according to 
teacher focus group participants “makes two scheduled observations per classroom each year and that’s about 
it.” Because the faculty is stable, he apparently feels that they are doing a very “good” job and does not need 
to hover over them. 

Teachers feel the principal is handling the administrative paperwork and duties and can be depended 
upon to handle major problems. As one teacher put it, “if something major comes up, he will be there. He 
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will come find you on duty if necessary.” They also feel that the principal is supportive and stands behind 
his teachers, applauding him for “getting between” the parents/community and teachers. 

There is little upheaval apparent at Bayou Elementary and few apparent problems to manage; as a 
result, detailed school-level policies are not so evident here as they might be in other settings. For example, 
the staff do not appear ruled by rigid disciplinary policies. Teacher remarks during the focus group suggest 
that disciplinary problems are primarily handled at the classroom level and are referred to the principal only 
as a last resort or in the event of repeated behaviors. However, more coordination may be needed because, 
as one teacher noted during the focus group, “Most of the time, teachers don’t know if a behavior is a first 
offense.” 

Feedback from the principal interview and teacher focus group suggest that the principal makes most 
of the decisions at the school, particularly in regard to budgeting, school partnerships, etc. There is very little 
faculty turnover at Bayou -- usually from the occasional retirement or teacher moving out of the parish. 
When vacancies do occur, the central office fills them based on seniority. Though work has begun on a five- 
year improvement plan, the process is progressing slowly, and many topics addressed during the site visit 
are addressed in the school’s own five-year plan. 

As is evident from Mr. Boudreaux’s survey responses, he is very proud of Bayou Elementary and 
gives uniformly high ratings to the climate, quality of instruction, and academic norms at the school. In fact, 
his mean attitudinal responses on the attitudinal survey are much higher than those of the students, teachers, 
or parents. 

The principal’s mean response on items relating to “safe and orderly environment” is quite high 
(4.40), nearly a full point higher than the mean student (3.45) and teacher (3.50) responses. His mean 
attitudinal responses are similarly high (4.40) for the school’s quality of instruction and academic norms, 
and are again substantially higher than the other three groups of respondents. 

The principal’s apparent contentment with these three aspects of life at Bayou Elementary may 
explain why, with the exception of serious problems, he takes a “hands off” approach to the instructional 



component. As alluded to before, teacher focus group respondents said that they handle most discipline 
problems themselves, at the classroom level. 

On three other facets of school life, the principal’s mean attitudinal responses are higher than any 
other group surveyed : parent/school relationship (4.20), organization/collaboration (3.20). However, his 
expectations for long-term student outcomes are lower than any other group (3.40 as compared to 3.59 for 
teachers, 3.81 for parents, and 4.05 for students themselves). 

Teachers feel the principal is handling the administrative paperwork and duties and can be depended 
upon to handle major problems. As one teacher put it, “If something major comes up, he will be there. He 
will come find you on duty if necessary.” They also feel that the principal is supportive and stands behind 
his teachers, and they applaud him for “getting between” them and the parents/community. 

As an organizational leader, however, he receives lower ratings. Teachers rate organizational 
leadership at the school quite low (2.78), a full point lower than the principal’s mean response (3.80). Their 
perceptions of organizational leadership at the school may reflect their apparent disappointment with 
opportunities for staff development (mean teacher response, 3.65) and collaboration (2.91). 

Teachers also feel the principal should interact more with students and create more opportunities to 
reward and acknowledge students for their accomplishments. “Some of the little ones here don’t know who 
he is,” one teacher commented. Another agreed and said, “The kids aren’t at ease around him because they 
seldom see him.” The principal’s lack of visibility apparently is not a new issue at the school; it reportedly 
is mentioned in the school improvement plan. 

The need for greater principal visibility that was cited in the teacher focus group also surfaced 
indirectly in the student focus group discussion. That is, few comments were made of the principal outside 
of his role as school disciplinarian. 

Though the SEAP survey instrument gives students no opportunity to comment on leadership at the 
school, parents give organizational leadership a lower mean rating than any other survey component (3.31 
on a scale of 5.0). Their next lowest rating is 3.64 for “safe and orderly environment,” an area of 

O 

ERIC 



30 

174 



responsibility that generally is attributed to the principal. The parents’ responses to these two items do not 
seem to stem from dissatisfaction with the parent/school relationship; they rate that aspect of the school 
environment relatively high (3.96), though not as high as the principal’s rating (4.40). 

Two PTA sponsored events, a Fall Halloween Carnival and a Spring Talent Show, are eagerly 
anticipated and well-attended. However, there appears to be no organized plan for parent volunteers within 
the school. Volunteers do not help in the classroom as instructors, though they sometimes “keep” classes 
for short periods of time. The limited roles that parents fill as volunteers reportedly involve running off ditto 
sheets, working on the school camival/talent show, or manning monthly candy/bake sales. 

Recommendations for Improvement 

The following eight recommendations were made for Bayou Elementary. These recommendations 
were considered in light of the school’s needs assessment, along with information from the on-site school 
analysis. 

1. The district should evaluate the physical plant at Lafourche School #3 to provide the students and 
teachers a more adequate environment & leam and work. The facilities at Bayou Elementary do not 
meet the needs of a K-7 school serving nearly 400 students. The district’s over-reliance on T- 
buildings to meet the school’s space requirements has left the staff little or no flexibility in assigning 
classroom space. Furthermore, there are no large spaces under roof where teachers can combine 
classes for instructional purposes or students can congregate during inclement weather. The one 
large covered area (the combination auditorium/cafeteria) is furnished in such a way that the meeting 
space is under-utilized. That is, the lunchroom is equipped with large tables and free-standing chairs 
rather than folding table/chair units that could be easily stored. 

2. Classroom assignments should be redistributed to reduce excessive student movement, especially 
in the upper grades . The layout of the current campus is too inconvenient for so much movement 
to be taking place. The school improvement team might consider the following strategies. 

(a) Cluster classes taught at the same or similar grade levels in a particular part of campus 
rather than distributing them all around the school site; 

(b) assign self-contained classes (for example, second or third grade classes in which students 
stay with the same teacher all day) to T-buildings and move departmentalized grades to the 
classroom annex or main building; 

(c.) wherever possible, ask teachers in the departmentalized grades to trade classrooms for part 
of the day rather than moving more than 30 children every hour. For example, 25 students 
could study English in T-Building 5 during first period; the social studies teacher assigned 
to T-Building 3 could trade rooms with the English teacher during second period so that the 
students could have two hours of back-to-back instruction without changing rooms. 

3. Ihs faculty should experiment whh cooperative teaching Jo make instructional deUvegy mors 
equitable and effective from one class or grade lo the next . It was pointed out, for instance, that 
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some students at Bayou receive P.E. more regularly than others because some teachers enjoy 
teaching P.E. more than others. For example, at one school in Barataria Parish, staff compensate 
for teachers’ differing interests by sharing teaching responsibilities; that is, one teacher will take 
along a colleague’s class when she takes her own class to P.E., freeing up time for the other teacher 
to plan. The second teacher returns the favor by combining the same two classes for art. The 
students benefit because they receive more enthusiastic instruction on a more regular basis, and the 
teachers benefit from increased planning time. 

More student recognition activities should bg initiated- The student focus group discussion suggests 
that students are highly motivated by the rewards and incentives their respective teachers provide 
for academic achievement and good behavior, yet the site team found relatively little evidence of 
any coordinated student recognition activities. Such a recognition program need not entail a great 
deal of expense. Student successes (and not just in academics areas, either) could receive more 
prominent attention on bulletin boards around campus. Rewards might also include extra time for 
a favorite activity or special status for a specified period of time (one school in Comite Parish, for 
example, allows its ablest older students to mentor younger students or serve as official campus 
greeters for a day or week). The students both enjoy and benefit from the increased recognition and 
their newfound responsibility. 

Steps should he taken to increase community and parental involvement in the SgllgQl- If not already 
in practice, these strategies should be considered. 

(a) Include parents as well as business and industry partners on the school improvement team; 

(b) develop business and industry support groups for school programs (e.g., agriculture and 
shipping industries); and, 

(c.) schedule open houses at the school twice a year to display student work and improve public 
relations with the community. 

Children in the u pp er grades should be given access to the gatSSI gXPlPJCation opportunities lYEkaily 
found in a comnrehensive middle school bv offering rotating vocational gkctjyes at lh£ mill and 
seventh grade levels . These electives could include the exploratory courses of Agriscience, Business 
Education, Family and Consumer Science (Home Economics) and Technology Education (Industrial 
Arts). Some course explanations can be found in Bulletin 741, Minimum Standards for State 
Approval in Reimbursed Programs in Vocational Education (See sections 122-145. 1). Vocational 
program managers at the Department of Education can provide technical assistance in setting up 
these courses. 

The ship building and ship transportation industries that are located in Delta Parish have adopted 
schools in other parts of the parish in hopes of employing students after graduation. These courses 
will help students study careers in shipping and transportation industries and better prepare them for 
career paths they will follow in high school. These courses will also reinforce the academic 
standards by providing practical applications of academic subjects. Courses could be taught by 
itinerant teachers from other middle/junior high schools in the district. 

Vocational student organizations are integral parts of the curriculum of each of these courses 
(Agriscience, Future Farmers of America; business education. Future Business Leaders of America; 
family and consumer science. Future Homemakers of America; and technology education. 
Technology Student Association). Such organizations will offer students intra- and extra-curricular 
activities, and will teach skills of leadership, organization, competition, and teamwork. Students 
also will receive recognition through these student organizations. While students are enrolled in 
these rotating electives, academic teachers would have the opportunity to have a planning period 



where they could collaborate. Elective teachers and academic teachers could collaborate at 
scheduled faculty meetings before or after school hours. 

7. Football or some other competitive athletic program should be instituted if a survey of students, 
faculty, and parents demonstrates general support far such a program- It was obvious from the 
discussion in both the student and teacher focus groups that the termination of the football program 
is perceived to have limited the social development opportunities for boys and girls. Presumably, 
an athletic competition also would give the surrounding community another means for relating to 
the school. 

8. A ppropriate steps should be taken & identify a SUS&S22QL £21 lh£ current principal (who plans In 
retire at the end of next school year) and to appoint that person as administrative assistant during this 
final year pf the principal’s service Ip Ihs system . This would provide a logical and smooth 
transition to a different administration because it would (a) enable the administrative assistant to 
become familiar with the new system, and (b) make for a comfortable "changing of the guard." The 
collaboration and teamwork approach to administration would foster cooperation and would provide 
a model of collaborative leadership that could possibly be the impetus for shared decision-making 
at the school level. 

The recommendation for improvement in the administrative collaborative component is to find a 
replacement for Mr. Boudreaux and make him/her the administrative assistant for the last year of Mr. 
Boudreaux’s service to the system. This would provide a logical and smooth transition to a different 
administration. The change would allow the interested participants to become familiar with the new system 
and make for a comfortable “changing of the guard”. The collaboration and teamwork approach to the 
administration would foster cooperation and would provide a model of collaborative leadership that could 
possibly be the impetus for shared-decision making at the school level. 

Creole Elementary School 

Atchafalaya Parish is irregularly shaped, located in south-central Louisiana. It is about 30 miles 
south of Baton Rouge and 60 miles west of New Orleans. The southernmost tip is about 25 miles from the 
Gulf of Mexico. The parish encompasses 365 square miles, and according to the U.S. Census, had a 
population of 22,859 in 1996. Bayou Lafourche is the parish’s principal waterway, and many of the parish’s 
residents live, work, and attend school along LA 1 or 308, the state highways that skirt the bayou’s western 
and eastern banks. 

Creole Elementary is located in a community in the extreme northern part of the parish. The school 
faces LA Highway 1 . It is bounded by a com field on one side and a church and cemetery on the other. The 



community is hamlet-sized and very rural. The focus of the community is characterized by its thirteen 
churches. Many of the students’ parents work either in the sugar cane fields or the local sugar cane mill. 
Parental involvement in the school is low in part because of limited access to transportation. 

Atchafalaya Parish ranks below the state average both financially and educationally. For example, 
28.2% of all persons in the area live below the poverty level, whereas the state average is 23.6%. Some 
3 1.3% of households had average annual incomes of less than $15,000 in 1990 as compared to a state 
average of 36.3%. Almost half (49.6%) of the adults in Atchafalaya Parish do not have a high school 
education as compared to 3 1 .7% for the state. The economic and educational performance indicators are not 
surprising in this region as residents have traditionally earned their livelihood in the agricultural, fishing, and 
petroleum industries in which semi-skilled workers were long able to earn good wages. Therefore, it is not 
surprising that in 1990, 44.9% of the parish’s workforce were blue collar workers (U.S. Bureau of the 
Census). 

Of risk factors for children and youth, Atchafalaya Parish has 17.6% of its households headed by 
single parents in 1990 as compared tol9.1% statewide. The parish’s teen pregnancy rate was 18.2% as 
compared to a statewide total of 1 7.6%. 

The Atchafalaya Parish system employed nearly 304 full-time classroom teachers in 1995-96 at an 
average annual salary of $24,985 compared to a statewide average of $26,800 (Bulletin 1472, Annual 
Financial and Statistical Report, 1995-96). Data from Table 1 indicate that the school has a slightly smaller 
percentage of its faculty with a master’s degree or higher (30%) than the district (33%). 

Atchafalaya Parish ranks 39th in the state in enrollment and number of schools, with 4,879 students 
in grades PK-12 enrolled in the district’s twelve public schools in 1995-96. The district compares favorably 
with the state in annual per pupil expenditure. In 1995-96, the district spent $4,576 per student in average 
daily membership compared to a state average if $4,468 (Bulletin 1 472, Annual Financial and Statistical 
Report, 1995-96). Public school students here outnumber nonpublic students more than 13 to 1. The 
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parish’s two nonpublic schools enrolled approximately 358 students in grades PK-8 in 1995-96 (Bulletin 
1472, Annual Financial and Statistical Report, 1995-96). 

Creole Elementary serves pupils in grades PK through 4. Of the pupils attending Creole Elementary 
in 1995-96, 10.5% were special education students, with none identified as “gifted/talented” since screening 
for gifted and talented is not part of the school’s program. The percentage of special education students is 
slightly above the state average, while the gifted/talented population is obviously under-represented relative 
to the state average of 2.7%. 

The percentage of at-risk Title I students reported for this school in 1993 was 82.3%, increasing to 
91.5% of the pupils in 1996-97 who are eligible for the free/reduced lunch program. Because of the 
socioeconomic status of the student body, the school qualifies under the Improving America s Schools Act 
(P.L. 1 03-3 82) as a Title I school. 

Tenure of the Principal and Other Administrators 

The principal at Creole Elementary , Ms. Hebert, in Spring 1997 had been in place for two years. 
She had previously taught special education for eighteen years in the parish school system. 

Ms. Hebert appeared focused on acquiring resources, providing more professional development for 
her faculty, increasing parental and community involvement, increasing pupil motivation and self-esteem, 

\ and providing effective teaching to all types of learners. The principal was seen in many areas of the school 

campus during the SEAP site visit. She appeared knowledgeable about all aspects of school life. 

The assistant principal has been at Creole Elementary for twenty-eight years. He was a teacher at 
the school prior to assuming his administrative position. He was visible around the school during the site 
visit, primarily in his role as chief disciplinarian. The pupils seem to like him. He did not appear to be 
involved in instructional leadership activities during the SEAP site visit. 

The principal at the time of the site visit said that Creole Elementary “has a lot of work to do to turn 
itself around.” She had a low overall opinion of the school’s current academic status and felt that instruction 
needed improvement. The school’s pupils score below the national norm on achievement tests, but the 

35 

O 

ERIC 



179 



principal believed that they could change and improve. The principal believed that several teachers were 
making concerted efforts to use innovative teaching techniques. Faculty at Creole Elementary used the 
Assertive Discipline Policy in their classrooms and elsewhere on the school grounds. When in-school 
suspensions associated with the program were given, the pupils knew the consequences. The principal 
encouraged the “Harry Wong approach” of practicing proper routines and classroom procedures. She 
scheduled Dr. Wong to speak to her faculty about his philosophy on effective teaching and has used his 
Effective Teaching tapes and workshops in professional development sessions at the school. 

The principal has encouraged teachers to write grants, and the school has received several which 
were used to buy educational materials. The school has obtained a LEARN grant for $7000. Title II funds 
for math and science and Title IV funds for libraries and supplies are also available. The principal also has 
received a grant with LSU to integrate technology into the classroom. The principal usually made the 
decision on how resources were spent, though she used teacher input in her decision-making. Additionally, 
the district office has provided the teachers a $600/year grant to use for classroom materials. 

The principal has developed community partners for the school, including persons from the local 
sugar cane mild, a local grocery, and the local churches. The churches helped to provide transportation to 
the schools. 

By having more than 50% of its pupils on free/reduced lunch, the school qualifies to become a Title 
I schoolwide program. However, the SEAP observation team saw no indication that such a program was 
planned or in operation at the school in 1996-97. 

The principal of Creole Elementary during the Spring 1997 gave the school rather low scores in four 
areas associated with the overall academic status of the school: a score of 1 .0 on the item rating the “General 
Reputation” of the school in the community; a score of 2.6 on the scale measuring the academic norms at 
the school; a score of 3.0 on the scale measuring expectations for pupils’ success and academic ability; and 
a score of 3.2 on the scale measuring the quality of instruction at the school. The principal’s ratings on these 
four scales are among the lowest recorded during the SEAP statewide site visits in the Spring 1997. 



The principal’s attitudes are consistent with the comments that she made during her interview. She 
had a low overall opinion of the school’s current academic status and felt that improvement was needed in 
the quality of instruction. Her rating of 1 .0, the lowest rating possible, on the school’s “General Reputation” 
indicates the degree to which she felt the school needed to improve its image in the community. However, 
the principal’s ratings on the other scales (i.e., collaboration, leadership, staff development, job satisfaction) 
were all 4.0 or better, indicating more positive attitudes in these areas. 

The teachers rated the overall organizational leadership at Creole Elementary (the principal and 
administration) at 3.97, which is relatively high given their overall low rating of the school’s reputation (only 
2.08). The teachers rated collaboration between the faculty and administration at 3.6, which was slightly 
below the principal’s rating of 4.0. Overall, the teachers appeared to have a higher opinion of the principal 
than they did of several other aspects of the school. In the focus group and in informal conversations, the 
teachers at Creole expressed respect for the principal as a professional and indicated they felt she was 
working extremely hard to improve the school. 

All respondents rated the scale measuring how “safe and orderly” the school was at 3.4 to 3.6, which 
indicates some satisfaction and some dissatisfaction with that component of the school. This, along with 
“Academic Futility 3 ” (at 3.5 to 4.0), were the only attitudinal measures that were generally consistent across 
all respondent groups. 

A few teachers negatively remarked to the site visitors that they felt the principal had singled them 
out to write grants and assume other leadership roles, which made them feel overworked. 

The principal indicated that some of the teachers were “very territorial” and rarely shared resources 
or ideas with other teachers. She attributed this to a lack of resources at the school and to the fact that the 
teachers were trying to protect what they had. Additionally, several teachers at the school appeared to want 
to maintain their traditional teaching techniques rather than use some of the innovative techniques that the 
principal had tried to implement. 
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Pupils were positive about Creole, based upon the attitudinal responses. They rated expectations for 
pupils’ academic success and ability, academic norms, and quality of instruction higher than any other group 
(teachers, principal, parents). Mean pupil attitudinal scores on these scales were all above 4.0. 

The pupils scored the scale measuring how “safe and orderly” the school was lower (3.38) than any 
other group, although the scores of the other groups were very similar. The pupils, faculty, principal, and 
parents all perceived the school as lacking some components of good “order” or discipline. In the focus 
group and in informal conversations, some pupils indicated that the “good” pupils were unfairly punished 
with the “bad” ones. They also indicated that they didn't like the fighting that occurred among some of the 
pupils. 

Pupils expressed the highest sense of academic futility of the groups, a score of 3.5 on a scale on 
which five indicated “lack of futility.” Several of the students expressed the desire to go into the Armed 
Services when they graduated from high school to escape negative home and community situations. 

Parents rated the “General Reputation of the School” 3.43, which was significantly higher than the 
1 .0 rating from the principal or the 2.08 rating from the teachers. In general, the parents’ ratings were similar 
to those of the teachers and students in several areas, particularly, safe and orderly environment, academic 
norms, academic futility, and quality of instruction. The parents had higher expectations for the pupils (3.81) 
than either the teachers (2.53) or the principal (3.0). The lowest rating from the parents was 3.31 for 
organizational leadership, which was considerably lower than the ratings from either the teachers (3.97) or 
the principal (4.6). The largest discrepancy between the ratings of parents and teachers occurred on the scale 
measuring the parent/school relationship: parents rated this a relatively high 3.96, while teachers rated it 
only 2.53. The parents’ rating is in line with that of that of the principal (4.0) and the pupils (3.99). 
Apparently, the teachers perceive a more negative relationship with parents than do any of the other groups. 
This result, coupled with the discrepancy in pupil expectations that exists between the parents and teachers, 
indicates areas of potential conflict between the two groups. 




38 

182 



Recommendations for Improvement 

Recommendations for improvement were arrived upon by the SEAP team and the needs assessment 

completed by the school. The following seven were made for Creole Elementary: 

(1) Inservice training aimed at raising the teachers! expectation levels fk the students should te 
undertaken. The average score for teachers’ expectations for their students (2.53) was considerably 
lower than either the parents’ (3.81) or the students’ (4.43), and among the lowest recorded in the 
SEAP site visitations during Spring 1997. High expectations for students has been a cornerstone of 
the research on effective schools for two decades, and it should be emphasized at Creole Elementary. 
There are a variety of inservice programs available aimed at raising teacher expectations, and the 
school improvement team should select (or develop) one that best suits the specific context at 
Assumption School #2. 

(2) Inservice training aimed at improving the faculty^ perception of the general repytattin fifths sshoal 
should also be undertaken . This recommendation is a corollary of Recommendation #2. The 
faculty’s rating of the general reputation of their school at 2.08 was among the lowest recorded in 
the SEAP site visitations in the Spring 1997 and was considerably below the perception of the 
parents (3.43). Teachers need to believe that their school is a good school, as well as believe that 
their students can learn. Again, there are a variety of inservice programs aimed at increasing the 
self-esteem of faculties, and the school improvement team should select (or develop) one that best 
suits the specific context at Creole Elementary. 

(3) Focus on improving the fourth grade NRT scores, especially in the areas of reading comprehen sion. 
language expression, and mathematics concepts and applications. The school improvement team 
needs to establish some long-term academic goals for the school, and addressing deficiencies in 
these specific skill areas appears to be a good place to start. Of course, reading comprehension is 
key to future achievement in school, so this area may be the most important one initially. 

(4) Provide staff development for all teachers in hands-on and interactive myi & present kssgPS - The 
low score in mathematics concepts and application at the fourth grade level may indicate the need 
for more hands-on and interactive methods in the classrooms at Creole School. According to the 
observers, most instruction at the school was text based and focused on the whole class. Classroom 
observational data revealed weaknesses in instructional practices in the areas of interactive learning, 
adjusting for individual differences, and higher order thinking skills. There was little evidence of 
hands-on learning or the use of a variety of materials in the delivery of instruction. Seeking out 
programs and techniques which build upon interactive teaching methods would move the focus of 
teaching away from the teacher and more toward the students. 

(5) Focus staff development on specific teacher skill areas , especially at the first gBd£ Igysi- First grade 
is obviously crucial to the success of pupils, and LCET results indicate that there is a need for 
improvement in several skill areas at this grade level. Teachers at the second grade level may be 
used as models, since their LCET scores in several of these areas were higher than those of first 
grade teachers. Specific suggestions related to this recommendation include: 

(a) Provide time for teachers to observe effective teaching, allowing master teachers to serve 
as mentors, utilizing appropriate outside resources (such as audio-visual examples). 

(b) Behavior management plans should be provided to teachers who have difficulty with 
classroom management. 
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(6) The school improvement plan could also include some of the following recommendations: 

(a) Make sure teachers and parents understand the new content standards. Both the school 
and community must decide and agree on exactly what the students are to achieve. 

(b) Work toward improving the relationship between teachers and parents. 

(c ) Establish an extensive staff development plan emphasizing specific grade level content 
standards, cooperative teaching and learning methods, LCET attributes, etc. 

(d) Work on a better organization for recesses and better transitions from class to class as 
consistency and clarity appear to be lacking here. 

(7) Establish a functioning school improvement team ihsl will dgYSlgp SchPPlwid? goals fading to 
specific attitudinal. behavioral, and cognitive changes . There is an obvious discrepancy in 
perception between the teachers and the parents as to strengths and weaknesses of Creole 
Elementary. The new principal needs to establish a school improvement process (inclusive of the 
entire school community) that will result in common perceptions and goals. From the academic end, 
perhaps the principal can utilize the Title I schoolwide program to develop common academic goals 
among the faculty members. 

The recommendation for improvement in the administrative collaborative component is to establish 
a functioning school improvement team that will develop schoolwide goals leading to specific attitudinal, 
behavioral, and cognitive changes. There was evidenced from the attitudinal surveys a discrepancy in 
perception between the teachers and the parents as to strengths and weaknesses in Creole Elementary. Ms. 
Hebert, the new principal, needs to establish a school improvement process (inclusive of the entire school 
community) that will result in common perceptions and goals. From the academic end, perhaps the principal 
can utilize the Title I schoolwide program to develop common academic goals among the faculty members. 

The former principal at Creole Elementary encountered resistance in her efforts to initiate change 
at the school. Ms. Hebert should profit from this experience by allowing certain faculty members who do 
not wish to participate in school reform to “mark time” until their retirement. Obviously, the membership 
of the school improvement team is crucial to its success, so the principal should spend adequate time 
structuring it in a way that will increase its chance for success. 

CONCLUSION 

The integrated SEAP process (SEAP-I, SEAP-II, SEAP-III) assumes that context specific school 
improvement can be initiated through the intensive site based analysis associated with the mixed method 
school effectiveness research. A link is presumed between school effectiveness research and school 
improvement projects. While such a link is logical, many researchers in the past have worked exclusively 
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within either the school effectiveness or the school improvement paradigm. In fact, the SEAP process is an 
attempt to forge links between three separate research traditions: SEAP I operates largely within the school 
indicator area, SEAP-II within the school effectiveness research field (especially effective schools research), 
and SEAP-IH within the school improvement tradition. 

Leading scholars in the international arena are just now calling for such integrated programs 
involving school indicators, effectiveness research, and improvement efforts (e.g. Brown, Riddell, & 
Duffield, 1966; Fitz-Gibbon, 1996; Reynolds, Hopkins, & Stoll, 1993; STOL & Find, 1996; Willms, 1992). 
If SEAP progresses as planned, it will be one of the first, if not the first, large scale attempt at such an 
integrated approach. 

While the components of SEAP are linked through integrated databases and research staffs, it should 
be remembered that from a practical point of view SEAP is a two-track, parallel system of statewide 
accountability and local improvement activities. Even though school accountability and improvement are 
theoretically linked to one another, each can have very different, independent goals. 
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Tying School Improvement to School Accountability: 

A Review of the School Effectiveness and Assistance Pilot Study 

Phase HI (SEAP-IH) 



Abstract: This paper summarizes activities associated with school improvement projects in 12 
SEAP Cohort One schools during School Year (SY) 1997-98 and the plans for future 
improvement activities in SY 1998-89. Faculty and administrators at each of the 12 schools were 
supposed to voluntarily revise their current School Improvement Plans (SIPs), drawing on (a) 
recommendations from Louisiana Department of Education (LDE) reports based on SEAP-II 
findings from SY 1996-97 and (b) needs assessments obtained from the faculties as part of the 

SEAP-II process. 

Two state educational representatives [one from the LDE and one from a Regional 
Service Center (RSC)] were assigned to assist each of the 12 schools. Their assistance was 
initiated by the 12 Cohort One schools and was consistent with the goals of the SIPs at those 
schools. The 12 Cohort One school improvement projects were undertaken in the schools in SY 
1997-98 with varying degrees of success. Reasons for their relative success are discussed. 

The tentative plans for school improvement efforts in Cohort One and Cohort Two 
schools (an additional 45 schools) in SY 1998-99, and in future years, are discussed as a final 
topic in this paper. A twofold purpose for this long term school improvement plan is discussed: 
“front loading” improvement of classroom teaching and building capacity to implement 
comprehensive reform. 

SEAP site teams and external school improvement specialists can provide valuable 
external perspectives on the strengths and weaknesses of schools and facilitate the delivery of 
needed resources and services to support school improvement. At times, the SEAP process may 
even serve as a mechanism for validating the staff* s own preconceived needs. Ultimately, 
however, the direction and the impetus for improvement should come from the school itself, 
backed by the shared commitment of faculty, administrators, district, and community. 
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Tying School Improvement to School Accountability: 

A Review of SEAP-HI 

L Introduction 

The Louisiana Department of Education initiated a school improvement process in SY 
1997-98 as part of the second year of the School Effectiveness and Assessment Pilot. Since the 
emphasis in SY 1997-98 was on SEAP-H, an intensive school assessment process, activities 
associated with SEAP-HI, the school improvement process, were kept simple. 

There are six stages to the current improvement activities associated with SEAP-UI. 

(1) Intensive on-site analysis of 12 Cohort One schools in the Spring 1997 using the 
SEAP-II process. (Described in a previous paper given in this session.) 

(2) Generation of a SEAP-II School Report for each of the 12 Cohort One schools, 
including recommendations for school improvement. (Also described in a previous paper.) 

(3) Presentation of the SEAP-II reports first to the Principal and Superintendent and then 

to the faculty as a whole. (Ending in October, 1997) 

(4) Presentation of a School Improvement Conference for representatives of the 12 

Cohort One Schools. (November, 1997) 

(5) Refinement of School Improvement Plans by the 12 Cohort One Schools based on 
recommendations from SEAP-II reports. (This stage is voluntary.) 

(6) Deployment of two person school assistance teams (one from the Louisiana 
Department of Education housed in Baton Rouge and one from the Regional Service Center in 
which the school is located) to the 12 Cohort One schools. (Beginning in January, 1998) 

(7) Refinement of the SEAP-HI school assistance model for SY 1998-99 and future 

years, based on several types of inputs. 

The following report will include: a brief review of the recommendations from SEAP-H, 
a review of the process whereby schools were presented their SEAP-H report, a synopsis of the 
School Improvement Conference, a description of the experiences of the school assistance teams 
in the 12 Cohort One schools in SY 1997-98, and a description of a tentative plan for school 
improvement efforts in SY 1998-99 and future years. 
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IL A Brief Review of the SEAP-U Recommendations for Cohort One Schools 

As indicated in the previous paper (Heroman, Pol, and Franklin, 1998), the two most 
frequently occurring categories of SEAP-II recommendations for Cohort One schools were in. 

(1) leadership (typically associated with school level change) and 

(2) professional development (typically associated with changing the behaviors of 
teachers in classrooms). 

This two level approach to school improvement is consistent with the overall philosophy 
that has driven the SEAP process: meaningful school change must occur simultaneously at both 
the classroom and school levels. Therefore, assessment data must be gathered at both levels and 
recommendations for improvement must be made at both levels. 

Change at the School and Classroom Levels: A Brief Literature Review 

This philosophy of pursuing change at both the school and classroom levels came out of 
the school effectiveness and improvement literatures of the 1990’s. During the early 1990’s, 
several school improvement or “restructuring” models were widely adopted by schools and 
districts. Most reformers agreed that the main purpose of these school restructuring models was 

to transform teaching practice which in turn would lead to improvement in student learning 

(Elmore, 1995 and Finnan, 1996). Fullan (1993) indicated, however, that the "learning core", 
which consists of both instructional practices and teacher culture, is the most difficult area to 
change. Keller and Soler (1996) further reported that “deep changes in teacher behavior” are rare 
events, and when they occur, are a result of long-term internalization of beliefs and practices. 

Elmore (1995) called for further research into teaching practice in restructuring schools, 
suggesting that the relationship between structural school changes and changes in teaching are 
mediated by factors such as teachers’ skills and knowledge. Taylor and Teddlie (1992, 1996) 
presented such research in their analysis of a prominent restructuring district. Despite verbal 
support from top level district administrators, evidence indicated that restructuring had, in fact, 
not influ enced the classrooms. Teachers in schools classified as "highly restructured" were no 
more likely to collaborate with their colleagues than were teachers from low participation schools. 
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Teacher-directed and whole-group approaches prevailed in classrooms of both schools types (high 
or low participation in restructuring), with group work and team teaching occurring rarely. 

The link between school restructuring efforts and classroom behaviors in this highly 
restructured district had not occurred, and evidence from other studies indicates that this is not an 
isolated phenomenon. Another example comes from a study (Meza and Teddlie, 1998; Teddlie 
and Meza, 1 998) of two supposedly restructured schools with very different classroom teaching 
indicators (one highly effective, one highly ineffective). The restructuring approach that these 
schools had both followed was one of the well known “special strategies” (Stringfield, Millsap, 
and Herman, 1997), which had a well articulated, systematic framework for improvement 

The fact that two schools engaged in a similar restructuring effort differed on the quality 
of classroom instruction provides more evidence that partrcipating in a restructuring process does 
not always impact the instructional core (Elmore, 1995; Fullan, 1993 and Levin, 1996). This 
result indicates that the direct assessment of classroom teaching quality should occur early in the 
school improvement process. Thrs study also indicates that the Principal is essential in any 
improvement effort (Christiansen, 1996 ; Crone & Teddlie, 1995 and Teddlie & Stringfield 1993), 
since it was mainly differences in the two Principals’ behaviors that led to the large differences in 
teachers’ classroom behaviors. 

The fail ure to find evidence of improved teaching in some restructured schools stands in 
contrast to research that has linked effective teaching with effective schooling. While the teacher 
effectiveness and school effectiveness literatures evolved separately, several studies have fruitfully 
merged the methods from these two areas over the past decade (Teddlie, 1994). Researchers 
conducting sophisticated school effectiveness research (e.g., Brookover, et fll, 1979, Mortimore, 
et al. 1988; Rutter, et al. 1979; Stringfield, Teddlie and Suarez, 1985) began exploring classroom 
processes during the 1970's and 1980's, due to dissatisfaction with the explanatory power of 
extant economic and sociological models. These researchers used informal observations and 
survey proxies for teacher effectiveness variables, and they were able to explore aspects of the 
schooling process that had not been previously examined in school effectiveness research. 

Starting in the mid- 1980's, researchers working within the school effectiveness research 
paradigm began explicitly including classroom observations (and consequently teacher 



effectiveness variables) in their research (e.g., Creemers, etal, 1996; Crone and Teddlie, 1995; 
Stringfield and Teddlie, 1991; Stringfield, Teddlie, and Suarez, 1985; Teddlie, Kirby, and 
Stringfield, 1989; Teddlie and Stringfield, 1993; Virgilio, Teddlie, and Oescher, 1991). For 
example, Teddlie, Stringfield and their colleagues used the Stallings’ Observation System (SOS) 
and an instrument composed of variables gleaned from Rosenshine's (1983, 1986) reviews of 
teacher effectiveness research in their school effectiveness research. 

These studies of teacher effectiveness variables within the context of school effectiveness 
research revealed consistent mean and standard deviation differences in classroom teaching 
between schools classified as effective or ineffective. For example, results from Teddlie, Kirby, 
and Stringfield (1989) indicated that teachers in effective schools were more successful in keeping 
students on task, spent more time presenting new material, provided more independent practice, 
demonstrated higher expectations for students, provided more positive reinforcement, and so 
forth, than did their peers in matched ineffective schools. 

In addition to these mean differences in teaching behaviors between effective/ineffective 
schools, interesting differences in patterns of variation were also found. For instance, the 
standard deviations reported for teaching behavior were smaller in more effective as opposed to 
less effective schools. This result indicates that there are processes ongoing at more effective 
schools (e.g., informed selection of new teachers, effective socialization processes) that result in 
more homogeneous behavior among teachers in which the "trailing edge" of teaching is somehow 
eliminated. 

This reduced variance in teacher behavior associated with school effectiveness status is a 
dramatic illustration of the interaction between school and classroom variables. Some process, 
either selection or socialization or both, must be operating at the school level to result in the 
classroom level differences. 

Therefore, SEAP employs information from both the school and the class levels in its 
assessment and improvement processes, because current research in school effectiveness and 
improvement indicate that it is necessary to do so in order to engender true change. 
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Commonalities Among Leadership and Professional development Recommendations for the 
12 Cohort One Schools 

Tables 1 and 2 present a content analysis of the frequency of SEAP-II recommendations 
for school improvement across Cohort One schools (Heroman, Pol, and Franklin, 1998). Table 1 
presents the recommendations related to leadership, while Table 2 presents the frequencies related 
to professional development. It is interesting that there were so many similarities in SEAP-II 
recommendations across the 12 Cohort One schools, since they represented a cross-section of 
schools in Louisiana in terms of both demographics and effectiveness. 

The three most frequent leadership recommendations involved: 

(1) scheduling/planning/protecting academic time, 

(2) school and classroom discipline policy, and 

(3) classroom observations/feedback. 

The three most frequent professional development recommendations involved: 

(1) instructional methods, 

(2) individual needs/special needs students, and 

(3) classroom management. 

The remainder of this section involves a discussion of specific recommendations for each 
of these six leadership and professional development recommendations. These recommendations 
were excerpted from two of the twelve Cohort One schools (Schools A-3 and B-l). These 
excerpts are included in this paper to give the reader a better understanding of the nature of the 
SEAP-II recommendations. The recommendations are located in Table 3. 

Schools A-3 and B-l were somewhat different from one another. 

(1) School A-3 was a middle school in South Louisiana, which had a high percentage of 
low-SES students and which scored low on standardized tests. The school was in transition 
during the time of the SEAP-II visit , the Principal during SY 1996-97 was replaced in SY 1997- 
98, due to poor performance. The assessment team was aware of this as they wrote their report. 
They noted in the Recommendations section that: “The presence of a new Principal in SY 1997- 
98 will make this an important year for initiating change. The new Principal should use the school 
improvement team in these efforts.” 
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Table 1 

Frequency of SEAP-II Recommendations Across 
Cohort One Schools: The Leadership Dimension 



Leadership Category 


Number of Times Noted 
for Cohort One Schools 


Scheduling/Planning/Protecting Academic Time 


7 


School and Classroom Discipline Policy 


7 


Classroom Observations/Feedback 


4 


School Improvement Team 


2 


Improve Communication/Collaboration 


2 


Class Size 


2 


New Principal 


1 


Reward Pupil Success 


1 


Homework Plan 


1 


Special Education 


1 


Student Involvement 


1 


Total Recommendations Associated with Leadership 


29 
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Table 2 

Frequency of SEAP-II Recommendations Across 
Cohort One Schools: The Professional Development Dimension 



Professional Development Category 


Number of Times Noted 
for Cohort One Schools 


Instructional Methods 


7 


Individual Needs/Special Needs Students 


7 


Classroom Management 


4 


Structure Opportunities/Staff Development Coordination 


4 


Higher Order Thinking Skills 


2 


Technology, Discipline, and Data Analysis 


2 


Learning Styles 


1 


Teachers on Self-Esteem 


1 


Cooperative Teaching 


1 


Total Recommendations Associated with Professional 
Development 


29 
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(2) School B-2 was an elementary school in North Louisiana, which served predominantly 
middle-class students and which scored above average on standardized tests. The assessment 
team noted in the Recommendations section that “Although the instructional staff at School B-l 
has created and provides a generally positive learning environment, there is room for some 
improvement ” 

The example of Leadership Dimension 1, Scheduling/Planning/Protecting Instructional 
Time, came from School A-3: The administrators and faculty shou ld place a high priority on 
conserving instructional time . (See Table 3 for more details.) 

The example of Leadership Dimension 2, School and Classroom Discipline Policy, also 
came from School A-3: The school’s discipline policies and reg ulations should be relevant and 
applied consistently . (See Table 3 for more details.) 

The example of Leadership Dimension 3, Classroom Observations/Feedback, came from 
School B-l . The Principal/instructional leader should increase classroom visitations an d 
corresponding teacher evaluations . (See Table 3 for more details.) 

The example of Professional Development Dimension 1, Instructional Methods also came 
from School B-l. Faculty should be encouraged to u se new curricula and instructional 
techniques In conjunction, the variety of instruction s h ould be increased to accommpda te 
different learning styles among pupils . (See Table 3 for more details.) 

The example of Professional Development Dimension 2, Individual Needs/Special Needs 
Students, came from School A-3: Instructional delivery should be diversified to accommodate the 
individual needs of all learners . (See Table 3 for more details.) 

The example of Professional Development Dimension 3, Classroom Management, came 
from School A-3 : Teachers should be encouraged to colla b orate in their classroom managemen t, 
instructional planning and delivery . (See Table 3 for more details.) 
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Table 3 

Excerpts from SEAP-II School Improvement Recommendations, 
Cohort One Schools A-3 and B-l 



Example of Leadership Dimension 1 - Scheduling/Planning/Protecting Instructional Time - 

School A-3 . 

Thp administrators and faculty should place a high priority on conserving instructional 

time . Teachers cannot teach and pupils cannot concentrate when instructional time is abbreviated 
or segmented by interruptions. The school improvement team should take a look at the school 
schedule in order to identify and recover lost time. Special consideration could be given to: 

(a) restricting use of the public address system to schoolwide announcements (as opposed to 
announcements aimed at specific individuals, but broadcast schoolwide), and limiting its 
use to one (or at most two) 6r;'e/broadcasts each day; 

(b) reducing time lost during transitions to and from lunch ; and 

(c) providing inservice training for teachers on classroom management, focusing on helping 
teachers improve transitions within instruction and make better use of instructional time. 

Example of Leadership Dimension 2 - School and Classroom Discipline Policy - School A-3 

The school’s discipline policies and regulations should be relev ant and applied 
consistently . There is a general sentiment among pupils and teachers that too much attention is 
paid to policing minor infractions, while more serious offenses carry inadequate penalties. This 
latter criticism particularly relates to misbehavior that is potentially embarrassing to the school and 
would result in more serious penalties for the pupil if reported to district staff There is a general 
perception among pupils that some staff show partiality when enforcing school rules, for some 
pupils consequences are minimal for misbehavior that would carry serious penalties for others. 

As a first step in improving the disciplinary climate, the Principal and staff should consider 
creating a committee to (a) review and revise the school code of pupil conduct, and (b) make 
professional development for rule enforcement. The committee should include representatives 
from the school administration, faculty, parents, and pupils each elected by peers. An external 
professional (e g., a child welfare and attendance officer from the central office or local juvenile 
justice professional) could prove a valuable resource to the committee and clarify the school s 
legal responsibilities relative to pupil discipline. Through deliberation, the committee could 
identify constructive ways to reduce misbehavior while encouraging pupil self-discipline. 
Discussions may also facilitate “buy-in” from those groups whose support is critical to the long- 
term enforcement of rules. 

Example of Leadership Dimension 3 - Classroom Observations/F eedback - School B-l 

The Principal/instructional leader should increase classroom visitations and corresponding 
teacher evaluations . The strong sense of collaboration between Principal and faculty should be a 
good foundation for school improvement. Teachers should receive regular feedback regarding 
their performance in the classroom, especially since there is a natural variance in this performance 
over time. 
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Example of Professional Development Dimension 1 - Instructional Methods - School B-l 

Faculty should be encouraged to use new curricula and instructional techniques. In 
conjunction the variety of instruction should be increased to accommo date different learning 
styles among pupils . The use of new technology in the classroom was limited and should be 
increased. Teachers should also receive training that will enable them to incorporate 
technological aids in their instruction. The training should go beyond the basic instruction for 
usage and should include techniques for incorporating technology efficiently and effectively in the 
classroom. An expanded emphasis on technology also will help teachers to increase the variety of 
materials used, thereby better addressing the needs of children who are auditory learners, visual 
learners, kinesthetic learners, etc. Teachers also would benefit from inservices aimed at improving 
their classroom management skills, especially with regard to maximizing the amount of time 
available for instruction. 

Example of Professional Development Dimension 2 - Individual Needs/Special Needs 

Students - School A-3 . . 

Tnctnirtinnal delivery should be diversified to accommodate th e individual needs of all 

learners . It appears that most of the instruction at School A-3 is whole-group instruction, and 
that teacher-directed discussion aimed at the average pupil is the primary method of instruction. 

In too many classes, brighter pupils appeared bored, waiting for their classmates to complete an 
assignment. Similarly, some pupils may fall behind or become disruptive, attempting to foUow a 
lesson that is too fast-paced. Inservices on various aspects identifying and accommodating 
individual differences among students should be part of the staff development activities at the 

school. 

Example of Professional Development Dimension 3 - Classroom Management - School A-3 

Teachers should be encouraged to collaborate in t heir classroom management, 
instructional planning and delivery . The site team found little evidence that teachers regularly 
meet to coordinate management techniques across classrooms or to coordinate their instruction 
from one discipline to the next or one grade to the next (e.g., planning lessons so that concepts 
learned in science reenforce concepts learned in math, or that content taught in fifth grade English 

is reinforced in sixth grade English). ... 

The teachers should solicit inservice training from the administration on classroom 
management, focusing on helping teachers improve transitions within instruction and make better 
use of instructional time. A coordinated behavioral management program across grade should be 
adopted or developed. 

Classes are scheduled in a manner that prohibits joint planning times. Therefore, 
rearrangement of the school schedule may be necessary to find time for coUaborative planning. 
For instance, the staff might consider a team-teaching approach to scheduling. That is, pupils 
could be divided into groups, with each group receiving all their instruction from a cross- 
disciplinary team of four to five teachers. The team could be assigned a common time to plan or 
discuss the needs of specific pupils (perhaps while the pupils are at physical education or lunch). 
Several middle schools in Louisiana are using the team-teaching approach with notable success. 
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ITT. A Review of the Feedback Sessions Held 
in Cohort One Schools, October, 1997 

The information contained in Sections III and IV comes from evaluations conducted at 
the School Improvement Conference for Cohort One Schools, November, 1997 (Teddlie, 1997). 
At that conference, Cohort One Principals and teachers were asked to complete evaluation forms 
concerning both the feedback sessions that had been held in their schools in October, 1997 and 
the School Improvement Conference they had just attended. 

The following section summarizes the participants’ responses to a survey assessing their 
reactions to the feedback that they had received in October regarding their school site visits from 
the Spring 1997. This survey was completed by 49 teachers and Principals at the end of the first 
day of the School Improvement Conference. Altogether the survey contained three closed-ended 
items and two open-ended items. 

Responses to the Closed-Ended Items Regarding the October Feedback 

The teachers. Principals, and central office personnel responded to the following closed- 

ended items: 

(1) Was the feedback regarding your school informative? 

(2) Was the feedback regarding your school accurate ? 

(3) Was the feedback regarding your school useful for im provement at your school? 

All these closed-ended items had five point response categories ranging from the most 

negative response (1) to the most positive response (5). For example, the response categories for 
the first item ranged from (1) not informative to (5) informative. Table 4 contains the means and 
standard deviations for the participants’ responses to the three closed-ended items. Altogether 49 
participants completed the evaluation forms. 

The average scores for all three closed-ended items was between 4 and 5 on the five point 
scale, indicating that the participants were generally pleased with the feedback. The highest 
average score was for the item concerning how informative the feedback was (4.449), while the 
lowest average score was for the item concerning how accurate the feedback was (4.102). 



12 



199 



Table 4 



Participants’ Responses to the Closed-Ended Items 
Assessing their Reactions to the October Feedback 



Item 


Number of 
Respondents 


Average Score of 
Respondents 


Standard Deviation 
of Responses 


1. Was the feedback 
regarding your school 
informative? 


49 


4.449 


0.937 


2. Was the feedback 
regarding your school 
accurate? 


49 


4.102 


1.085 


3. Was the feedback 
regarding your school 
useful for 

improvement at vour 
school? 


49 


4.429 


0.979 



Note . All these closed-ended items had five point response categories ranging from the most 
negative response (1) to the most positive response (5). For example, the response 
categories for the first item ranged from (1) not informative to (5) informative. 
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The standard deviations were moderate sized (around 1.00), indicating that there was 
some variance among the responses. The distribution of responses indicated that the majority of 
respondents (44) rated the items mainly 5s, with some 3s and 4s. A minority of respondents (5) 
rated the items 1, 2, or 3. More information on these negative responses will be included later in 

this section. 

Responses to the Open-Ended Items Regarding the October Feedback 

In addition to answering these closed-ended questions, the participants responded to the 

following open-ended items: 

(1) What was the most useful aspect of the feedback you received? 

(2) How could the feedback process be improved for other schools in the future? 

As noted above, altogether 49 participants completed the evaluation forms. Their open- 
ended responses were analyzed using the constant comparative method described by Lincoln and 
uba (1985), which involves unitizing and categorizing the responses. This constant comparative 
method was used to analyze open-ended data that will be described throughout this report, 
including the evaluations of the October feedback, the School Improvement Conference, and the 
experiences of the school assistance teams in the 12 Cohort One schools in SY 1997-98. 

The unitizin g aspect of the constant comparative method involves breaking down 
responses into the smallest pieces of distinct information. In this case, unitizing meant taking each 
person’s responses and breaking them down into the different useful aspects of the October 
feedback. Thus, the responses from some participants yielded multiple units of information (if 
they listed more than one useful aspect), while the responses from others yielded no units of 

information (if they did not respond to the question). 

Once the responses were unitized, they were then categorized. This involves giving a 
label or name to each of the distinct open-ended responses. For instance, if the respondent stated 
“The feedback gave us objective information regarding our school”, this response might be coded 
“Objective feedback”. 

Altogether, 59 separate units of information emerged from the categorization of the 
responses to the question regarding most useful aspects of the October feedback. Table 5 
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contains a summary of the responses to the question regarding the most useful aspects of the 
October feedback. 

The most frequently occurring response (33.9% of the total responses) was that the 
feedback session provided recommendations for improvement or mentioned specific areas that 
needed improvement. Typical responses included the following. 

“The information regarding the areas that need improvement. Our team now knows 

where to begin.” 

“It provided insights on the ‘specifics’ in our school that needed improvement.” 

“It gave us direction for our school improvement” 

Altogether, 56 separate units of information emerged from the categorization of the 
responses to the question regarding ways to improve the feedback in the future. Table 6 contains 
a summary of the responses to the question regarding ways to improve the feedback in the future. 

The three most frequently occurring responses were: 

(1) provide more specifics in the recommendations (17.9%), 

(2) give feedback before the visit, provide quicker feedback (12.5%), and 

(3) provide evaluations of individual teachers (10.7%). 

Altogether, these three categories accounted for over 40% of the total number of 
responses. Typical responses from these categories included: 

“More feedback on test scores and improving them.” 

“The recommendations could be more specific. Leave nothing to question or 
assumption.” 

“Presentation to the school staff of findings prior to on-site feedback session in order that 
staff can review and prepare for questioning.” 

“Provide immediate feedback.” 

“Be sure to let each teacher know their strengths and weaknesses individually.” 

“Give individual teachers their evaluation report.” 

Most of the categories In Table 6 are self-explanatory, but the “inaccurate feedback” 
category needs some explanation. While only three participants made this response, they 
represent a minority of school site personnel who were upset at the feedback they had received. 

15 




202 



Table 5 

Participants’ Responses to the Open-Ended Item 
Assessing their Perceptions of the Most Us eful Aspect 
of the October Feedback 



Category 


Number of Responses 


Percent of Responses 


Recommendations for 
Improvement, Specific Areas 
Needing Improvement 


20 


33.9% 


Objective Feedback, Outsider 
Point of View 


8 


13.6% 


Assessment of School’s 
Strengths 


7 


11.9% 


Assessment of School’s 
Weaknesses 


7 


11.9% 


Helped Us to Evaluate Our 
School, Initiate Discussions 
About Improvement 


5 


8.5% 


Pointed Out Differences in 
Perceptions 


4 


6.8% 


Provided Positive, Courteous 
Feedback 


2 


3.3% 


Information on How to Write 
Goals 


2 


3.3% 


Miscellaneous 


4 


6.8% 


Total 


59 


100.0% 



Note . Percentages may not add to 100% due to rounding. 
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Table 6 

Participants’ Responses to the Open-Ended Item 
Assessing their Perceptions of Wavs to Improve 
the Feedback Process in the Future 



Category 


Number of Responses 


Percent of Responses 


Provide More Specifics in the 
Recommendations 


10 


17.9% 


Give Staff Feedback Before 
the Visit, Provide Quicker 
Feedback 


7 


12.5% 


Provide Evaluations of 
Individual Teachers 


6 


10.7% 


Make Site Visits Earlier in the 
Year, Spend More Time in 
the Site Visits 


5 


8.9% 


Explain Questionnaires, 
Surveys 


4 


7.1% 


Inaccurate Feedback 


3 


5.4% 


Provide Recommendations in 
Areas That the School Can 
Control 


3 


5.4% 


Site Visitors Should Write 
Reports, Provide Feedback 


3 


5.4% 


Tone of Feedback Was Too 
Negative 


2 


3.6% 


Positive Comments 


2 


3.6% 


Miscellaneous 


11 


19.6% 


Total 


56 


100% 



Note . Percentages may not add to 1 00% due to rounding. 
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Based on conversations at the School Improvement Conference, these individuals came primarily 
from one school, with one or two persons from a second school. At the preliminary August 
meetings between LDE personnel, local superintendents, and Principals, the Principal from one of 
the 12 schools objected to the feedback received as being “inaccurate”. A typical response from 
this school is as follows: 

“I felt as other teachers in our faculty that a ‘true picture’ of our teaching strategies were 
not totally observed fairly. The visiting team came for a day and observed in our classrooms for 
30-45 minutes and may have seen the same teachers.” 

This feedback is very useful for SEAP personnel in terms of the development of the site 
visit process. Some tentative points may be made: 

(1) The feedback regarding the inaccuracy of the responses was definitely a minority 
opinion. Only one of the twelve Cohort One schools was unified in this response, with a second 
having some reservations. The other ten schools were in agreement that the feedback was 
accurate and useful. 

(2) The SEAP-II procedure involves sampling in several areas: the classroom 
observations, the parents selected to receive the surveys, the students and teachers selected for 
the focus groups. It is possible that some unintended biases may emerge from these sampling 
procedures. This is one reason why a draft report was given to the Principals in advance of the 
feedback visits, so that they could make suggestions for changes in the final report. This is also 
one reason why the school’s faculty was asked to complete needs assessments, so that the report 
could be written with the input of those who know the school best. 

(3) Feedback concerning perceived inaccuracy of SEAP-II school reports will probably 
become more common as the process is used to examine schools in crisis . The experiences with 
Cohort One schools should help LDE personnel in the future in terms of responding to individuals 
who perceive the SEAP-II reports to have inaccuracies. 

Changes to be Incorporated in Feedback Procedure As a Result of the Evaluations 

With regard to the feedback provided to the schools, the following actions will be taken, 
to the degree possible, in the next round of visits: 
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(1) Conduct the school visits earlier in the year. For the Cohort Two schools in SY 
1997-98, these visits are scheduled to occur in January-March. (This in fact has occurred: all 
SEAP-II site visits in SY 1997-98 were completed before April.) 

(2) Get feedback to the Principals and Superintendents more quickly. This feedback 
should occur more quickly in SY 1997-98 since templates of the reports have been prepared, and 
site team members will be able to follow a heavily prescribed outline, with much information 
already completed, as they write the reports. 

(3) Cohort Two faculties will be given preliminary feedback before the school visits, so 
that they can be better prepared to ask questions and respond to the report. 

(4) The surveys and questionnaires will be explained in more detail in the feedback to the 
faculties. 

(5) Individuals who conducted the site visits will be more involved in the writing of the 
reports and in the delivery of the feedback to the Cohort Two schools. 

(6) Perceived inaccuracies in the reports will be taken into consideration in the production 
of the final report and in the generation of the commendations and recommendations. 
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IV. A Synopsis of the School Improvement Conference 
for Cohort One Schools, November, 1997 



A School Improvement Conference was held for the twelve schools Cohort One schools, 
selected LDE and RSC employees (including those individuals who were on the school assiatance 
teams), LSU employees, and other interested parties in Baton Rouge on November 3-4, 1997. 

This training was intended to prepare the participants to conduct SEAP-III school improvement 
activities during SY 1997-98. 

Altogether 108 individuals attended the workshop, including: 49 Principals and teachers 
from the 12 Cohort One schools, 1 1 district central office personnel, 43 members of the LDE 
staff, and 5 others (from LSU, the state school boards association, representatives of teacher 
unions, etc.) 

The agenda for the School Improvement Conference included: 

(1) four team breakout sessions for the school site teams, (2) an overview of the current 
state accountability initiatives, (3) an overview of the school improvement process, (4) a 
presentation by Sam Stringfield, from Johns Hopkins University, on special strategies for school 
improvement, (5) presentations on using achievement data to make decisions and on developing 
measurable goals, (6) a presentation on examining attitudinal and behavioral data in developing 
school improvement plans by Charles Teddlie, from Louisiana State University, (7) a 
presentation on forming school improvement teams, (8) a presentation by Jim Meza, from the 
University of New Orleans, on school reform in Louisiana 1990-97, (9) presentations on 
implementing planned change through staff development and content specific school improvement 
strategies, (10) presentations on finding resources to support school improvement by LDE, SRC, 
and university staff, and (1 1) a separate breakout session for the Principals. The Principal 
session was added due to feedback received at the conference regarding the importance of getting 

the Principals together for networking purposes. 

Numerous training materials were also prepared and distributed at the meeting. 

The survey assessing the participants’ reactions to the School Improvement Conference 
was distributed at the end of the second day of the conference. The survey contained three 
closed-ended items and three open-ended items. This survey was completed by all conference 



participants, including school site personnel, central office personnel, and LDE personnel. 
Altogether 77 participants completed the evaluation forms. The remainder of the participants, 
primarily LDE employees who attended only one day of the conference, did not complete the 
evaluation forms. 

Responses to the Closed-Ended Items Regarding the School Improvement Conference 

The evaluation form contained the following closed-ended items: 

(1) Was the School Improvement Conference informative? 

(2) Was the School Improvement Conference useful? 

(3) Was the School Improvement Conference relevant to improv ement at vour school? 

All these closed-ended items had five point response categories ranging from the most 

negative response (1) to the most positive response (5). For example, the response categories for 
the first item ranged from (1) not informative to (5) informative. Tables 6-8 contain the means 
and standard deviations for the participants’ responses to the three closed-ended items, broken 
down into groups based on where the participant was employed. 

The average scores for all three closed-ended items for all groups was between 4 and 5 on 
the five point scale, indicating that the participants were generally pleased with the School 
Improvement Conference. The highest average score for the item concerning “how informative ” 
was the conference was given by the central office personnel (4.833), while the lowest average 
score was given by school site team members (4.256). The overall mean score across all 
participants was 4.481, with a standard deviation of 0.736. (See Table 7.) 

The highest average score for the item concerning “how useful” was the conference was 
given by the LDE personnel (4.708), while the lowest average score was given by the school site 
team members (4.230). The overall mean score across all participants was 4.455, with a standard 
deviation of 0.787. (See Table 8.) 

The highest average score for the item concerning “how relevant“was the conference to 
improvement at your school(s)” was given by central office personnel (4.833), while the lowest 
average score was given by the school site team members (4.230). The overall mean score across 
all participants was 4.346, with a standard deviation of 0.883. (See Table 9.) 
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Table 7. Participants’ Responses to the Closed-Ended 



Item Assessing How Infom 


native the Conference Was 


Group 


Number of 
Respondents 


Average Score of 
Respondents 


Standard Deviation 
of Responses 


School Site Team 


39 


4.256 


0.818 


Central Office 


6 


4.833 


0.408 


LDE Personnel 


24 


4.708 


0.550 


No Designation 


8 


4.625 


0.744 


Total 


77 


4.481 


0.736 



Note . The response categories for this item ranged from (1) not informative to (5) informative. 



Table 8. Participants’ Responses to the Closed-Ended 
Item Assessing How Useful the Conference Was 



Group 


Number of 
Respondents 


Average Score of 
Respondents 


Standard Deviation 
of Responses 


School Site Team 


39 


4.230 


0.872 


Central Office 


6 


4.667 


0.516 


LDE Personnel 


24 


4.708 


0.624 


No Designation 


8 


4.625 


0.744 


Total 


77 


4.455 


0.787 



Note . The response categories for this item ranged from (1) not useful to (5) useful. 



Table 9. Participants’ Responses to the Closed-Ended Item 
Assessing How Relevant the Conference Was to Improvement at Their School 



Item 


Number of 
Respondents 


Average Score of 
Respondents 


Standard Deviation 
of Responses 


School Site Team 


39 


4.230 


0.930 


Central Office 


6 


4.833 


0.480 


No Designation 


7 


4.571 


0.787 


Total 


52 


4.346 


0.883 



Note . The response categories for this item ranged from (1) not relevant to (5) relevant. 
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Responses to the Open-Ended Items Regarding the School Improvement Conference 

In addition to answering these closed-ended questions, the participants responded to the 
following open-ended items regarding the School Improvement Conference: 

(1) What was the most useful aspect of the School Improvement Conference? 

(2) What actions will you take as a result of this conference? 

(3) How could the School Improvement Conference be improved in the future? 

Altogether, 116 separate units of information emerged from the categorization of the 

responses to the question regarding the most useful aspect of the School Improvement 
Conference. Table 10 contains a summary of the responses to the question regarding the most 
useful aspects of the School Improvement Conference. 

The three most frequently occurring responses to this question were: 

(1) small group team meetings (25.9% of the total responses), 

(2) speakers (25.0%), and 

(3) focus on improvement, goal setting (19.8%). 

Thus, 70.7% of the respondents indicated that these three aspects of the conference were 
the most useful. Typical responses regarding the small group team meetings were as follows: 

“Small group sessions that could discuss 'real life' situations" 

"Sharing in small groups. At this conference I think our team was clearly able to see in 
what direction we need to move." 

"The small groups where we walked through the processes with lots of information." 

The speakers were seen as a strength of the conference, and several of them were 
mentioned by name. Typical responses regarding the speakers were as follows: 

"The information given by the speakers was very useful in developing our plan." 

"Having some 'new blood' share ideas with us." 

"Speakers were prepared, informative, and entertaining." 

The focus on improvement and goal setting was the third most often mentioned useful 
aspect" of the conference. Typical responses included the following: 

"As a participant with a limited knowledge of school improvement measures, I feel I am 
leaving with a broader understanding of the concept." 
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Table 10 

Participants’ Responses to the Open-Ended Item 
Assessing their Perceptions of the Most Useful Aspect 
of the November 3-4 School Improvement Conference 



Category 


Number of Responses 


Percent of Responses 


Small Group Team Meetings 


30 


25.9% 


Speakers 


29 


25.0% 


Focus on Improvement, Goal 
Setting 


23 


19.8% 


Networking, Bonding 


9 


7.8% 


Facilitators 


5 


4.3% 


Meeting with State, Regional 
Personnel; Meeting with 
School Administrators 


5 


4.3% 


Information on Data Analysis 
Applied to Improvement 


4 


3.4% 


Resources for School 
Improvement 


2 


1.7% 


Negative Comments 


2 


1.7% 


Miscellaneous 


7 


6.0% 


Total 


116 


100% 



Note . Percentages may not add to 100% due to rounding. 
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"Helped address strengths and weaknesses as focal points for goals and objectives." 

"The most useful aspect of the conference was the information presented in helping with 
creating goals." 

, Responses to the open-ended question asking "What actions will you take as a result of 

this conference?" were broken into two sets of categories: those for the school site personnel 
(Table 11) and those for the other participants, such as central office staff and LDE employees 
(Table 12). As indicated in Table 11,51 separate units of information emerged from the 
categorization of the responses to this question by the school site personnel. 

The three most frequently occurring responses to this question were: 

(1) develop improvement plans and goals, revise improvement plans (25.5% of the total 
responses), 

(2) involve other faculty in process, more teamwork, revise improvement committee 
structure (21.6%), and 

(3) implement improvement process (17.6%). 

Thus, 65% of the respondents indicated that these were the actions that they would take 
as a result of attending the School Improvement Conference. All of these responses had to do 
with directly starting or improving a school improvement plan. Typical responses regarding 
developing improvement plans and goals or revising improvement plans included the following: 

“Meet as a team at the school level to target goals and objectives.” 

“We will reexamine our school improvement plan so it can conform with ideas presented 

here.” 

Typical responses regarding involving other faculty in the process, promoting more 
teamwork, or revising the improvement committee structure included the following: 

‘Take back information to fellow faculty members.” 

“Reorganize school improvement committee structure.” 

“We hope to create two goals regarding reading. We will do this in a cooperative, small 
‘team’ group setting.” 

Typical responses regarding implementing the improvement process included the 
> following: 
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Table 11 

School Site Personnel’s Responses to the Open-Ended Item 
Assessing their Perceptions of the Actions That They Will Take as a 
Result of the November 3-4 School Improvement Conference 



Category 


Number of Responses 


Percent of Responses 


Develop Improvement Plans 
and Goals, Revise 
Improvement Plans 


13 


25.5% 


Involve Other Faculty in 
Process, More Teamwork, 
Revise Improvement 
Committee Structure 


11 


21.6% 


Implement Improvement 
Process 


9 


17.6% 


Develop a More Enthusiastic 
and Open Approach 


5 


9.8% 


Request and Employ 
Resources for Improvement 


4 


7.8% 


More Staff Development 


3 


5.9% 


Miscellaneous 


6 


11.8% 


Total 


51 


100% 
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“Bring back to school and put into action plans we came up with.” 

“Put into action our goals.” 

As indicated in Table 12, 40 separate units of information emerged from the categorization 
of the responses to the question asking about actions “that they will take by other personnel 
(central office personnel, LDE personnel). The four most frequently occurring responses to this 
question were: 

(1) provide technical assistance to districts, schools (37.5% of the total responses), 

(2) hold meetings regarding information, disseminate materials (12.5%), 

(3) prepare to provide technical assistance (10%), and 

(4) provide resources for school improvement (7.5%). 

The final open-ended question asked “How could the conference be improved in the 
future?” As indicated in Table 13, 93 separate units of information emerged from the 
categorization of the responses to this question by the participants. 

The most frequently occurring suggestion for improving the School Improvement 
Conference was to have “more small groups interaction, more discussion” (23.7%). Typical 
responses regarding this suggestion included the following. 

“Additional time in small group meeting. Team likes to share and problem solve. 

“Not enough time for individual team meetings.” 

“Longer group time to implement improvements.” 

“Spend more time discussing problem analysis and problem definition before the groups 
work on goal setting.” 

No other suggestion for improvement was made by more than 10% of the participants. 
The next five most frequently made suggestions were: 

(2) better facilities (9.7%), 

(3) revise schedule (7.5%), 

(4) need a follow up workshop, need meetings at school (7.5%), 

(5) provide further opportunities for networking (6.5%), and 

(6) too much time in presentations (6.5%). 
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Table 12 

Other Participants* Responses to the Open-Ended Item 
Assessing their Perceptions of the Actions That They Will Take as a 
Result of the November 3-4 School Improvement Conference 



Category 


Number of Responses 


Percent of Responses 


Provide Technical Assistance 
to Districts, Schools 


15 


37.5% 


Hold Meetings Regarding 
Information, Disseminate 
Materials 


5 


12.5% 


Prepare to Provide Assistance 


4 


10.0% 


Provide Resources for School 
Improvement 


3 


7.5% 


Adopt a School 


1 


2.5% 


Miscellaneous 


8 


20.0% 


Positive Comments 


4 


10.0% 


Total 


40 


100% 



I 
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Table 13 

Participants’ Responses to the Open-Ended Item 
Assessing their Perceptions of Wavs to Improve 
the November 3-4 School Improvement Conference 



Category 


Number of Responses 


Percent of Responses 


More Small Groups 
Interaction, More Discussion 


23 


24.7% 


Better Facilities 


9 


9.7% 


Revise Schedule 


7 


7.5% 


Need a Followup Workshop, 
Need Meetings at School 


7 


7.5% 


Provide Further Opportunities 
for Networking 


6 


6.5% 


Too Much Time in 
Presentations 


6 


6.5% 


Include Other Personnel 


4 


4.3% 


More Time with Facilitators 


4 


4.3% 


Prepare Participants for 
Conference 


1 


1.1% 


Positive Comments 


11 


11.8% 


Miscellaneous 


15 


16.1% 


Total 


93 


100% 
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Changes to be Incorporated into the Conference as a Result of the Evaluation 

It is unclear what type of School Improvement Conference will be held for Cohort Two 
schools in the Fall 1998, since there will be a much larger number of them (45). However, if such 
a conference(s) is held for Cohort Two schools, the following changes may be made: 

(1) Schedule more small group sessions, since the Cohort One site team members found 
them so beneficial. 

(2) Schedule a breakout session for the Principals, since the addition of this to the Cohort 
One conference was so successful. 

(3) Provide more formal mechanisms for the schools to network after the improvement 
conference. 

(4) Send the participants more information about the conference in advance of the 
meeting, so they can better prepare for it. 

(5) Hold the meeting in a larger facility. 
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IV. A Description of the Experiences of the School Assistance Teams 
in the 12 Cohort One Schools in SY 1997-98 

The following section summarizes the responses of 17 individuals (10 LDE employees, 
seven RSC employees) to a questionnaire concerning their contacts with schools during the 
school improvement phase of the SY 1997-98 SEAP program. These participants had been 
assigned, as members of an assistance team, to 12 schools that had been intensively assessed the 
previous year (SY 1996-97) using SEAP-II procedures. Information in this section has been 
taken from an evaluation report based on responses to the questionnaire (Teddlie, 1998). 

Two state employees (one from the LDE and one from the RSC in the region in which the 
school was located) were scheduled to visit each of the schools during the early part of 1998. 
They were to assist the schools in any way possible with the SEAP recommendations and with 
their School Improvement Plan (SIP). The nature of the assistance was to be determined by the 
school, not by the LDE and SRC employees, who were to make themselves available to the 
school personnel for whatever assistance they (the school personnel) deemed appropriate. 

A meeting was held on February 26, 1998 at the LDE to solicit feedback from the school 
assistance teams regarding their activities with the 12 Cohort One School Improvement Teams. 
This meeting involved each team verbally summarizing its visits to the Cohort One schools and 
completing a questionnaire concerning those visits. 

Methodology 

The questionnaire that each assistance team completed included three closed-ended items: 

(1) Is there an active school improvement process ongoing at the school you visited? 

(2) What kind of impact did the SEAP process (the intensive school assessment, the fall 
school improvement conference, your visits) have on the improvement efforts at the school you 
visited? 

(3) Did the school representatives that you met on your visit actively solicit further 
assistance from you in their ongoing school improvement process? 

There were also four open-ended items: 

(1) Who met with you during your visit to the Cohort One school in January or February 



1998? When did the meeting occur? 

(2) Describe the meeting that you had with the representatives at your school. 

(3) Describe the school improvement activities that you have undertaken (or will 
undertake) at your school this year (SY 1997-98). 

(4) What should the Department of Education and the Regional Service Centers do to 
assist Cohort One and Cohort Two schools in their improvement activities for SY 1998-99? 

The three open-ended items were analyzed using simple descriptive statistics (means, 
standard deviations, and ranges of scores for each of the items. The four open-ended items were 
analyzed using the constant comparative method described in an earlier section of this report. 

There were 17 members of the 12 Cohort One school assistance teams: 10 LDE 
employees and 7 SRC employees. Several of the team members, especially the SRC employees, 
were assigned to two or more schools. At the time of the February 26, 1998 meeting, reports 
were completed on only nine of the 12 schools. The other three schools had not been visited due 
to scheduling conflicts. Team members completed one questionnaire together for each school. 

Results from the Closed-Ended Items 

The mean response to the item concerning whether there was an active school 
improvement process at the school they visited was 3.89 on a five point scale, on which a 
response of “5" indicated a “very active process” and “3" indicated a “somewhat active process”. 
The responses to this item included: two “5's”, four “4's”, and three “3's”. (See Table 14.) 

The mean response to the item concerning the impact the SEAP process was having on 
the improvement efforts at the school they visited was 3.22 on a five point scale, on which a 
response of “5" indicated a “large impact” and “3" indicated a “medium impact”. The responses 
to this item included: two “5's”, two “4's”, two “3's”, two “2's”, and one “1". This wide range of 
responses indicates the importance of the individual differences among the schools in the study. 

The mean response to the item concerning whether school representatives actively 
solicited further assistance from the team members was 3.11 on a five point scale, on which a 
response of “5" indicated “a lot of assistance” and “3" indicated “some assistance”. The 
responses to this item included: two “5's”, four “3's”, and three “2's” . 
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Table 14 

Responses of the School Improvement Teams 
to the Closed-Ended Items 



Item 


Mean 


Standard 

Deviation 


Range of Responses 


(1) Is there an active school 
improvement process ongoing 
at the school you visited? 


3.89 


0.78 


3-5 


(2) What kind of impact did 
the SEAP process have on the 
improvement efforts at the 
school you visited? 


3.22 


1.39 


1-5 


(3) Did the school 
representatives that you met on 
your visit actively solicit 
further assistance from you in 
their ongoing school 
improvement process? 


3.11 


1.16 


2-5 



Note . Responses to item (1) ranged from 1 (not at all active) to 5 (very active). Responses to 
item (2) ranged from 1 (not impact at all) to 5 (large impact). Responses to item (3) 
ranged from 1 (they want no assistance) to 5 (they want a lot of assistance). 




33 



220 



Results from the Open-Ended Items 

Table 15-18 contain a summaries of the information from the four open-ended items after 
the responses had been analyzed using the constant comparative method described above. 

As indicated in Table 15, the LDE/SRC School Assistance Team met with the Principal on 
eight of the nine scheduled visits. The only time the Principal did not meet with the school 
assistance team was due to a “major mix up on the part of the Principal”, in which the Principal 
had mistakenly attended another meeting and had not informed his staff. The Assistant Principal 
met with the school assistance team instead, and another site visit was scheduled. 

Teachers were involved in six of the nine meetings; there were as many as six teachers in 
two meeting and as few as one in two other meetings. The Assistant Principal attended three of 
the meetings, while the entire School Improvement Team (or planning committee) attended three 
meetings. Some parents attended one of the meeting, while a district central office supervisor 
attended another. 

From the composition of the school representatives attending the school assistance 
meeting, it could be concluded that a good faith effort was being put forward in at least five of the 
nine schools. These schools had the following persons attending the meeting: (1) Principal, 
teacher, central office supervisor, randomly selected teachers (met informally after the scheduled 
meeting); (2) Principal, Assistant Principal, and six teachers; (3) Principal and planning 
committee; (4) Principal, Assistant Principal, and four members of the School Improvement 
Team; and (5) Principal, planning committee members, and parents. 

Only the Principal attended two of the school meetings, and only the Assistant Principal 
(in the place of the Principal) attended at a third school. Furthermore, the she faculty members 
who attended at another school were “picked by the Principal” and seemed “happy with the status 
quo” according to the school assistance team. 

The LDE/SRC assistance team provided more information regarding the SEAP school 
improvement recommendations at all nine schools, as indicated in Table 16. Details about what 
the school was doing with regard to their school improvement plans was forthcoming at seven of 



Table 15 

Summary of Responses to the Item Concerning 
Who the LDE/SRC School Assistance Team Met with During 
Their Visits to the Cohort One Schools 



School Representatives 


Number of Times the School Representatives 
Met with the LDE/SRC Representatives 


Principal 


8 


Teachers 


6 


Assistant Principal 


3 


School Improvement Team, Planning 
Committee 


3 


Parents of Students 


1 


Central Office Representative 


1 



Table 16 

Topics Discussed During 
Visits to the Cohort One Schools 



Topic Discussed 


Number of Times this Topic Was Discussed 


LDE/SRC Representatives Provided More 
Specifics Regarding the SEAP School 
Improvement Recommendations 


9 


School Representatives Described their 
Activities Related to their School 
Improvement Plan or Title I Schoolwide 
Program 


7 


Participants Discussed Loss of Teaching 
Time, Off Task Behavior, Etc. 


3 


Participants Discussed Follow-up to 
Prescribed Staff Development Activities 


2 


Participants Discussed Other Specific Aspects 
of School Improvement 


5 



O 
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the nine schools. Thus, these scheduled school site visits could be characterized, in general, as 
two-way communication opportunities for both the school assistance team and the local school 
improvement representatives to exchange information regarding recommendations for school 
change and what had been ongoing thus far in terms of that change. Specific problems with 
teacher performance in the classroom and staff development activities were topics that emerged in 
several of the schools. 

Responses by the LDE/RSC school assistance teams indicated that they had already 
initiated some type of school assistance in six of the nine schools before the scheduled site visit, as 
indicated in Table 1 7. The types of assistance included: giving demonstration lessons, providing 
resources, conducting workshops on school improvement plans, and providing assistance on new 
state testing plan. The list of resources provided was lengthy, including: research based practices, 
videos, standards based lesson plans, resource lists, and statewide school improvement plan 
formats. Of the three schools that had not been directly contacted before the site visit, two 
requested information of some sort that the school assistance team was to provide them in the 
future. The remaining school was the one in which the Principal had forgotten the meeting. 
Assistance to this school was “on hold” until a formal meeting occurred. 

The final open-ended question asked the school assistance teams to suggest what the 
LDE and the RSCs could do to assist Cohort One and Cohort Two schools in their improvement 
activities for School Year 1998-99. (Cohort Two schools are being intensively assessed during 
the School Year 1997-98.) The responses to this question are summarized in Table 18. The 
four most frequent responses to this item were: 

(1) Increase contact with schools by calling and conducting monthly meetings to discuss 
school improvement efforts. Use a structured protocol in these meetings. (Suggested by four 
school assistance teams.) 

(2) Provide ongoing technical assistance (including staff development) and provide 
resources for continuous improvement as requested by the school site improvement team. 
(Suggested by four school assistance teams.) 

(3) Provide more demonstrations of “hands on” solutions to teaching deficiencies. 
(Suggested by two school assistance teams.) 
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Table 17 



Descriptions of School Improvement Activities Undertaken 
by LDE/SRC School Improvement Teams at Cohort One Schools, 

SY 1997-98 



School Improvement Activity 


Number of Times this Specific School 
Improvement Activity was Described 


Visited Schools for Specific School 
Improvement Activity 


6 


Sent School Improvement Materials (e g., 
research based practices, videos, standards 
based lesson plans, resource lists, statewide 
school improvement plan formats) 


4 


Gave Demonstration Lessons 


2 


Conducted Workshop on Writing School 
Improvement Plans 


1 


Provided Assistance on Increasing Parent and 
Community Interactions with the School 


1 


Provided Assistance on New Statewide 
Testing Program 


1 
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Table 18 

Suggestions for What the LDE and SRCs Can Do to Assist SEAT 
Cohorts One and Two in their School Improvement Activities 
During SY 1998-99 



Suggestion 


Number of Times the Suggestion Was Made 


Call and Conduct Monthly Meetings to 
Discuss School Improvement Efforts. Use 
Structured Protocol. Increase Contact with 
Schools. 


4 


Provide Ongoing Technical Assistance 
(including Staff Development) and Provide 
Resources for Continuous Improvement as 
Requested by the School Site Improvement 
Team. 


4 


Provide More Demonstrations of “Hands On” 
Solutions to Teaching Deficiencies 


2 


Have LDE/SRC Assist in Writing/Revising 
School Improvement Plans. Include SEAP 
Recommendations in SIPs. 


2 


Provide Information on Funding Sources for 
School Improvement Efforts. 


1 


Document the Process of LDE/SRC 
Assistance in a Standardized Manner. 


1 


Develop Method for Informing the District 
Superintendent of the Results of the 
Improvement Efforts. 


1 
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(4) Have the LDE and SRCs assist in the writing and revising of School Improvement 
Plans. Include SEAP recommendations in the revised SIPs. (Suggested by two school assistance 
teams.) 

Conclusions 

The verbal reports and written responses to the questionnaires indicated that: 

(1) There was a sizeable impact of SEAP-II assessments on three of the schools, who 
were looking for considerable assistance from the LDE/SRC assistance teams. 

(2) The SEAP-II assessment had some impact in four other schools, who were also 
looking for some assistance from the LDE/SRC assistance teams. 

(3) The SEAP-II assessment had little impact in two schools, who were also not really 
interested in assistance from the LDE/SRC assistance teams. 

There was a great deal of variance among the schools, and within the districts, in their 
responses to SEAP. The questionnaire responses indicated that the SEAP-II intensive 
assessment process had at least a medium impact on the school improvement processes at six of 
the nine schools. The three schools where this did sot occur may be characterized as follows: 

(1) One school had a Principal who was about to retire and who did not want any 
meaningful change to occur at his school. He had “handpicked” teachers for the scheduled 
meeting with the school assistance team who were satisfied with the status quo. This happened 
in a district with a superintendent who was willing to make changes and illustrates the power of a 
Principal to effectively, and unilaterally, block school reform. 

(2) One school had a first year Principal, Assistant Principal, and secretary. The 
Principal indicated that he had been trying to straighten out procedural matters up until the time of 
the school assistance team’s visit and was ready to begin “work on the instructional aspect” of the 
school. This school had just begun writing its school improvement plan. The school assistance 
team believed that the Principal would eventually seek out help from the LDE/SRC team. 

(3) The third school was the one where the Principal had forgotten the meeting. This 
was also a first year Principal in a district where the Superintendent placed a lot of emphasis on 
continuous school improvement. The school assistance team indicated that the school 



administration wanted a lot of assistance; they just hadn’t got organized enough to request it. 

There was also a great deal of variance among the schools with regard to their response 
to SEAP-IH, the school assistance part of the program, which was just being piloted in SY 1997- 
98. The questionnaire responses indicated that six of the schools were asking for at least some 
assistance from the LDE/RSC school assistance teams in their ongoing school improvement 
processes . The three remaining schools, where this was rated a “2" , may be characterized as 
follows: 

(1) The first school was the same one characterized above as having a Principal who was 
about to retire and who did not want any meaningful change to occur at his school. 

(2) The second school was the same one characterized above as having a first year 
Principal and staff that were just beginning to write its school improvement plan. 

(3) The third school also had a first year Principal, but she had very definite ideas about 
the direction in which she wanted to take her school. The school assistance members wrote that 
“We left with an understanding that they will call when they need assistance.” It is interesting to 
note that a change in leadership was one of the recommendations that was made in the SEAP-II 
report for the school in the previous year. For this action, at least, the SEAP-II recommendations 
had been followed. 

While there were some negative responses to the SEAP-II recommendations and SEAP- 
IH school assistance teams in SY 1997-98, the majority of the school responses were positive. 
(Generally speaking, seven of the nine schools that were visited responded positively.) There 
were, in fact, some surprises in terms of the positivity of the response. The best example of this 
was a school which had responded very negatively to the SEAP-H feedback in the Spring 1997. 
This negativity had carried over into the November, 1997 school improvement meeting. Past that 
point, changes began to occur as the RSC school assistance member began to interact with the 
Principal and staff. The school’s personnel decided that while the SEAP-H report was still 
“flawed”, there was some merit to it and began to work on certain aspects of suggested school 
change. The LDE/RSC assistance team gave a response of “5" to the closed-ended “impact” 
item on the questionnaire, indicating that the SEAP process had a large impact on the 
improvement efforts at the school. 
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The variance across the schools’ responses to the SEAP-IU improvement process in SY 
1997-98 indicates the need for more consistency in the following years. To a certain degree, this 
was to be expected, since the SEAP-DI activities for SY 1997-98 were limited due to the 
emphasis on refining the SEAP-II process. One positive note regarding the SEAP-IU process for 
SY 1 997-98 concerns the interaction between the school assistance team members. At the 
February 26 meeting of the LDE and SRC team members, several spoke of the positive 
interactions that had occurred between them and also noted that they had learned from one 
another. The LDE personnel were particularly vocal about learning a lot about school 
improvement from the RSC personnel. 

A number of suggestions for improving the process emerged from this evaluation: 

(1) Name the LDE/RSC assistance team as soon as possible. 

(2) Have the RSC person involved in the final stages of the writing of the SEAP-EE 
report, so that they can be more familiar with the findings and have an influence especially on the 
writing of the recommendations. 

(3) Start the school assistance process sooner, probably right after the faculty meeting in 
which the commendations and recommendations are presented (in September or October 1998). 

(4) The school LDE/RSC assistance team should ask the Principal and School 
Improvement Team to bring their plans for addressing (or reasons for not addressing) the SEAP- 
II recommendations to the first scheduled meeting of the two groups. 

(5) The SEAP-II recommendations should be seen as an enhancement to the School 
Improvement Plan that already exists at the school. 

(6) The LDE/RSC assistance team should come to the first meeting with the School 
Improvement Team with a list of all available resources and services relevant to the SEAP-II 
recommendations. 

(7) There should be regional school improvement conferences in the early Fall 1998 
catering to the schools in the RSC regions who have gone through the SEAP-II process in the 
Spring semester 1998. 
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IV. Plans for School Improvement 
in SY 1998-99 and Future Years 



The overall plan for whole school improvement for the SEAP schools is designed with a 
twofold purpose. The first objective of this school redesign model targets improvement in 
classroom teaching behavior. This approach is founded on the assumption that the main purpose 
of restructuring schools is to transform teaching and learning. Improving schools can be thought 
as bringing the structure of the classroom in conformity with best available knowledge about 
teaching and learning. Transforming teaching practice in turn will lead to improvement in student 
learning (Elmore, 1995). The second purpose of the SEAP plan develops capacity at the school 
site to support these effective teaching practices through the implementation of comprehensive 
school reform. Comprehensive school reform is a dramatically different approach to school reform 
that focuses on reorganizing and revitalizing the entire school, rather than on isolated piecemeal 
reforms. This improvement strategy uses well-researched and well documented models of school- 
wide change supported by external technical assistance. Schools engaged in comprehensive school 
reform have challenging academic standards, engaged teachers, and strong parental and 
community support (United States Department of Education, 1998). 

“Front Loading” Improvement in Classroom Teaching Behavior 

Most researchers agree that it takes approximately five years to restructure a school 
(Levin, 1992). Meza & Teddlie (1998) report that deep changes in teaching practice are rare 
events and that instructional practice is the most difficult to change. The researchers further report 
that schools participating in a similar restructuring effort differed in teacher effectiveness. 
Evaluation reports of a Louisiana statewide restructuring effort indicate high levels of 
improvement in school contextual variables and process outcomes, such as, student attendance, 
school discipline, parental involvement and overall school climate. These outcomes occur 
frequently, and in some cases, during the first year of the change process. The studies also 
indicate improvement in academic achievement for the restructuring schools, however, there is 
very limited evidence of improved student achievement during the early years of the restructuring 
effort (Meza, Kennedy & Teddlie, 1997; Oescher, Brooks, & Meza, 1996). 
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The SEAP school improvement plan is an attempt to change this pattern of delayed and 
inconsistent outcomes in academic achievement for restructuring schools. The SEAP school 
improvement plan is designed to “front load” changes in teaching and learning through intensive 
professional development. Emphasis will be placed on developing special strategies for teaching 
disadvantaged children. Guided by the findings of the SEAP assessments, the SEAP schools, with 
assistance from the district and state, will develop professional development plans for the teachers 
and Principals. This approach to school improvement uses the teacher as the change agent. These 
teachers become the catalyst and energy behind transforming teaching practice in the school. 

Building Capacity for Comprehensive School Reform 

The second part of the SEAP school improvement plan consists of building capacity to 
improve the readiness level of the school for implementation of comprehensive school reform. 
Capacity building for the SEAP schools centers on three themes, 1) exploration and buy-in, 2) 
alignment, and 3) professional support systems. 

Exploration and Buy-In 

To assure SEAP schools are better prepared to implement a comprehensive change 
process, exploration and buy-in by the school communities is the first step in building capacity. 
Horsley & Kaser (1998) suggest that school change participants examine the values inherent in 
the proposed changes and specify ways that the change represents a good fit between the school 
or school district’s values. The SEAP schools’ full staff (administrators, teachers, para- 
professionals, and parent representatives) will carefully explore research based and effective 
comprehensive school designs. The schools, with assistance from the Louisiana State Department 
of Education and the school district, will consider a full range of comprehensive school reform 
models and determine which model is the best “fit” for their school. Agreement by 80% of the 
school community is needed before a model is selected. This high acceptance rate ensures that the 
school community is committed to implementing the design successfully. A written statement by 
the school district superintendent, supporting the decision of the SEAP school community, is an 
essential step in the early stages of building capacity for school wide change and establishing a 
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collaborative relationship between the school and the district’s central office. 

Alignment 

The state and district expectations and administrative procedures governing SEAP 
schools need to be aligned to support the school’s priorities in the restructuring process. Areas of 
alignment include; policies and reporting procedures, funding, and professional development. 

The Louisiana State Department of Education, local school districts, and SEAP schools 
will work collaboratively to align state and district policy and amend administrative regulations to 
support the SEAP school improvement processes. One example of this collaborative effort is for 
the district and state to accept the SEAP school improvement plan as the Title I school 
improvement plan annually request by the district and state. A second example of regulatory 
alignment is for the district to use teacher and Principal evaluation instruments that are inclusive 
of teacher and Principal’s strength and weaknesses which are reported in the SEAP assessment 
findings. 

Odden (1995) indicates that teaching all students to high standards is a goal that may not 
be achievable with the way schools are fiscally managed today. In the SEAP plan, funding at the 
state and district level will be aligned and allocated to support the priorities established by the 
SEAP school improvement plans. One strategy is the state and district can use to support SEAP 
schools is to cluster federal, state, and local financial resources. Funding, including , Title I, Goals 
2000, technology grant awards. Comprehensive School Reform Demonstration Program, can be 
fenced under the umbrella of school improvement and allocate these funds to priorities of the 
SEAP schools. A second strategy is for the state, local school district, and school site to 
reallocate existent funding, specifically those monies currently dedicated to non-instructional 
areas, to professional development and other school improvement priorities. Reallocation of 
funding is a strategy that may be particulary effective for many schools with high concentrations 
of students from low-income families (Odden, 1995). 

The Louisiana State Department of Education and local school districts of the SEAP 
schools will align professional development for Principals and teachers consistent and focused on 
the priorities established by the SEAP schools and the findings of the SEAP external assistance 



school visits. This professional alignment includes state level training, such as, the Principal 
leadership training and workshops offered by the state’s regional service centers. 

Professional Development Support Systems 

Haslam (1995) indicates that professional development is the cornerstone of school 
transformation. Professional networking builds on the strengths of the diverse experiences of 
members of the SEAP schools. Interactions with colleagues in other SEAP schools will also 
provide revitalization to sustain the hard work of school change. District and statewide 
networking opportunities, such as, grade level meetings, school visits, and periodic Principal 
meetings, will be offered to the SEAP schools. These professional meetings will provide 
opportunities for collaborative work, directly tied to improved performance for students, with 
colleagues in the SEAP network. 

Schedule of implementation of SEAP schools’ improvement plan: 

Summer and Fall, 1998 Principals and Teachers begin intensive professional development 

on special strategies in teaching disadvantaged children. 

Fall, 1998 SEAP schools conducts exploration and buy-in to a comprehensive 

school reform model. 

Spring, 1999 School initiates implementation of comprehensive school reform 

process. 
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